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SUMMARY 


Previous use of various task analysis processes for the purpose of display 
interface design or enhancement has run the risk of failing to improve user performance 
due to the analysis resulting in only a sequential listing of user tasks. Adopting an 
ecological approach to performing the task analysis, however, may result in the 
necessary modeling of an unpredictable and variable task domain required to improve 
user performance. Kirlik (in press) has proposed an Ecological Task Analysis 
framework which is designed for this purpose. It is the purpose of this research to 
measure this framework’s effectiveness at enhancing display interfaces in order to 
improve user performance. 

Following the proposed framework, an ecological task analysis of experienced 
users of a complex and dynamic laborattwy task. Star Cruiser, was performed. Based 
on this analysis, display enhancements were proposed and implemented. An 
experiment was then conducted to compare this new version of Star Cruiser to the 
original. By measuring user performance at different tasks, it was determined that 
during early sessions, use of the enhanced display contributed to better user 
performance compared to that achieved using the original display. Furthermore, the 
results indicate that the enhancements proposed as a result of the ecological task 
analysis affected user performance differently depending on whether they are 
enhancements which aid in the selection of a possible action or in the performance of an 
action. Generalizations of these findings to larger, more complex systems were avoided 
since the analysis was only performed on this one particular system. 
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CHAPTER I 

TASK ANALYSIS AND DISPLAY ENHANCEMENT 

System and display designers have often used the method of task analysis to 
assist them in the design or enhancement of displays. The task analysis provides the 
designer with information concerning the activities required of the people who will 
interact with the system at hand. By understanding what tasks these users will be faced 
with, the designer can attempt to develop a display, or enhance an existing one, which 
will assist, and not hinder, the user in performing the tasks. The exact steps in 
performing such a process may differ from designer to designer, but the goals are 
always the same. The designer hopes to create a human-machine system which will 
operate in a safe and efficient manner. 

In order to accomplish this goal, the system designer will generally begin the 
task analysis process by listing all functions in the system that involve, to some degree, 
human interaction. These functions are analyzed to determine what tasks must be 
performed by the human in order to fulfill the purpose of the function. Once these tasks 
have been identified, they are further broken down into the steps necessary to perform 
thencL Steps are then usually analyzed further to determine other factors which may 
influence the human’s performance in working with the system. Such factors include 
the stimuli which signal the onset and completion of the step; physical actions required 
to perform the step; decisions that must be made while performing the step; information 
required to complete the task; feedback information resulting from completion of the 
step; potential sources of error, and criterion for successful performance. The task 
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criticality and difficulty, as well as the number and skill level of the people required to 
operate the system, are often also estimated (Sanders and McCormick, 1987). 

This information can be especially helpful to the designer of human-computer 
interfaces. By analyzing the tasks required of the human in this manner, the designer is 
able to better determine what information the interface should provide to the user as 
well as what information, and actions, the user should provide to the system via the 
interface. In other words, the designer can decide what information will be inputted 
into the system and outputted by the system and how this transfer of information will be 
accomplished. 

Trying to simplify this process of performing a task analysis, attempts have 
been made to develop taxonomies of generic functions and actions performed through 
display interfaces. Williges, Williges, and Elkerton (1987) describe several such efforts 
including taxonomies which list common communication interfaces or user- system 
interface actions. The benefit of these listings is that they provide the designer with 
information concerning what information or actions should be accounted for in the 
display interface. How the information should be manipulated is usually left for the 
designer to decide. In order to develop useful, performance-enhancing displays though, 
designers need to be aware of how the presentation of information will affect user 
perfcmnance. 

The designer may receive help in making this determination if an ecological 
perspective towards the task analysis is adopted. Vicente (1990) describes such an 
approach as one which defines the system goals in terms of “the constraints in the 
environment that are relevant to the operator.” Vicente then proceeds to explain how 
traditional task analysis cannot account for variability in system behavior. This fault is 
due to the nature of the analysis being a “single, temporal sequence of overt behaviors.” 
The variability typically found in system behavior can result in the degradation of 
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system performance when its design has been based on such a sequence. Such 
variability results from changes in initial system conditions, unpredictable external 
disturbances, and differences in users' methods for operating the system. This 
variability is often not described by traditional task analysis well enough to ensure 
satisfactory system performance. 

Vicente (1990) states that separate descriptions of three classes of constraints 
are necessary in order to account fOT these sources of system variability. The first 
description is of the problem space in which the behavior occurs. The second is of the 
generic tasks that the operator is to accomplish. The final description is of the set of 
factors used in selecting possible actions that may be employed by different system 
operators in order to perform those tasks. By developing a complete set of descriptions, 
the designer is generally better able to predict the operator’s behavior for any given 
combination of constraints. As a result, the designer is more knowledgeable about the 
system environment and therefore can more closely accommodate the complex and 
dynamic world in which the system operates. 

Such an ecological approach to task analysis was taken by Kirlik (in press). It 
is his approach which was investigated in this research. Of concern is how effective 
such an approach to task analysis is at enhancing the display interface of a complex 
dynamic system in order to improve operator performance. A laboratory task known as 
Star Cruiser served as the system. Experienced users of this simulation were created 
to provide a basis for defining the functional problem space of Star Cruiser, the tasks 
performed during operation of the simulation, and the different methods, constraints, 
and rules that such users may employ during operation. This analysis follows Kirlik’s 
framework for ecological task analysis. Once the analysis was completed, areas of the 
current Star Cruiser interface which were possible causes of poor operator 
performance were highlighted. Based on these findings, enhancements to the display 
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interface were proposed and implemented. An experiment was then conducted to 
compare the enhanced display interface to the original interface by measuring at various 
levels user performance on the task. 

Upon completion of this research, it should be apparent as to whether or not 
this particular framework for ecological task analysis is beneficial and effective. If it is, 
then the degree to which it aids in enhancing display interfaces should be able to be 
determined. Possible results, therefore, would reveal that this framework is effective at 
enhancing the display for particular characteristics of the simulation. What those 
characteristics are can only be determined at the conclusion of this investigation. In 
addition, the framework may effect instances where training is occurring rather than 
normal operation. It should be constantly remembered, though, that this research is 
solely concerned with one particular ecological task analysis framework and its 
effectiveness on one particular systetiL Therefore, any conclusions made as a result of 
this research are context-specific. Though the findings may seem to be able to be 
applied to other frameworks or systems, any general theories concerning ecological task 
analysis may only be constructed once other frameworks and systems are investigated. 
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CHAPTER n 

ECOLOGICAL PSYCHOLOGY IN HUMAN-MACHINE SYSTEMS 

Some work has already been done in the field of ecological psychology in the 
context of human-machine systems. The two disciplines of ecological psychology and 
human-machine systems are considered well related since they both rely on analyzing 
and modeling the human-environment system as a unit (Kirlik, Miller, Jagacinski, in 
press). It is the goal of the ecological approach to human-machine systems to reduce 
the cognitive demands and errors attributed to hunuui control in complex systems. This 
goal is generally accomplished in two ways. The first method is to enhance the 
information presented to the user. The second is to improve the user's abilities at 
utilizing the presented information to determine control actions. The former is achieved 
by presenting the user with a nrore effective representation of the environment, 
generally through the use of computer information displays. The latter is the result of 
providing the user with assistance, possibly computer-generated, in determining what 
control actions to perform. As stated, some work along these lines has already been 
performed. As will be seen, even though different research e^orts share the name 
"EcologicaL” their approaches to improving the human-environment relation may be 
different Their goal, to improve human performance while reducing the cognitive 
demands of the user, is always the same. 

Throughout these discussions, it is important to bear in mind exactly what is of 
concern. A major focus of this research is to determine what properties of the 
environment usually information displays, will cause different cognitive activities. 
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usually resulting in some decision or action. Hammond summed up his attempt at 

answering this question in the following way: 

If (a) the displayed data present many redundant cues, (b) the cue values 
are continuous, (c) the cues are displayed simultaneously, (d) the cues are 
measured p^roperly, and (e) the subject has available no explicit., method 
for organizing this information into a judgement then intuitive cognition 
will be employed. Analytical cognition will be induced by the opposite set 
of conditions, (p. 310, Hammond 1990) 

Based on this theory, a relationship between the task and the required cognitive activity 
can be predicted. By understanding the above criteria and the type of relationship that 
exists between the two, the system designer can modify the information displays in 
whatever way is necessary to satisfy the criteria in order to achieve a desired effect. 

Two such procedures which incorporate these ideas, and more, are discussed here. The 
Ecological Interface Design and the Ecological Task Analysis frameworks are both 
designed with the intent to improve infcHmation displays and, thus, user performance. 
The former is concerned with identifying causes and solutions to rare-event occurrences 
in complex systems while the latter, the focus of the present research, is geared towards 
the specification of actions during system operation.. 

Ecological Interface Design 

Vicente and Rasmussen (1988) have developed a theoretical framework 
known as Ecological Interface Design (EID). Its foundation is based on the skills, rules, 
knowledge (SRK) framework as proposed by Rasmussen (Rasmussen, 1983). The 
purpose behind this framework is to design interfaces so that cognitive processes are not 
required at a higher level than the task necessitates. At the lowest level of the 
framework is skill-based behavior which involves highly automat^ sensory-motor and 
cognitive performances and occurs with little, if any, conscious control. The next level 
of the framework is rule-based behavior. This level is behavior activated by a 



7 


hierarchy of rules stored in working memory. Knowledge-based behavior comprises 
the highest level of the framework. It consists of behavior that is evoked when new, 
complex problems are encountered. These problems are tied to meaningful concepts 
through similar functional or physical properties. The SRK framework is used to 
ensure that users rely on lower levels of cognitive control (i.e., skill-based behavior). In 
addition, the framework also attempts to ensure that information presented via the 
display is of similar form to the method with which it is manipulated. The EID 
framework consists of three principles, one for each level of the SRK framework, in 
order to satisfy these goals. 

The first principle in the Rasmussen and Vicente (1988) framework is to 
permit the user to act directly on the display so that interaction with the system occurs 
"via time-space signals." This principle supports the skill-based behavior portion of the 
nnodel. Its purpose is to allow the user to manipulate what is displayed in the interface 
and is an attempt to develop a high-degree of manual skill within the user. 

The second principle is in support of the framework's rule-based behavior 
component. It states that a unique and "consistent one-to-one mapping" should exist 
between the underlying processes of the system and the perceptual information 
displayed on the interface. This information, in the forms of cues and symbols, is used 
to determine which actions arc proper to perform. By developing a unique and 
consistent mapping, it is possible for the user to merely rely on perceptual cues in order 
to control the system. This reliance on cues occurs instead of depending on knowledge- 
based behavior. By relying on such rule-based behavior, cognitive control of the 
system is reduced and thus mental effort is as well. In addition, because of the one-to- 
one mapping between the symbols and cues, the user can exhibit knowledge- based 
behavior while only having to rely on the cognitive loads typical of rule- based behavior. 
As a result, while experiencing such reduced cognitive load due to the rule-based 
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behavior, the user may continue to take advantage of the general applicability that is 
characteristic of knowledge-based behavior. 

Finally, the third principle involves presenting an "externalized mental model" 
to the user via the display interface. This principle is geared to facilitating knowledge- 
based behavior. By providing a model of the complex system to the user, the user no 
longer needs to keep track of the "complex casual network" they are reasoning about. 
Instead, the external model provides the support necessary for deductive reasoning 
about system states and planning actions. Vicente and Rasmussen (1988) also suggest 
that this model may additionally aid the user by providing the depth of understanding of 
system functionality required in order for error recovery. 

According to Flach and Vicente (1989), such an interface which 
accommodates all three levels of behavior in the SRK framework allows the user to deal 
with the entire range of task demands that arc encountered during interactions with a 
complex system (Flach and Vicente, 1989). By representing the display information to 
be in accordance with the cognitive processes of the user, the meaningful characteristics 
of the system become visible (Vicente, 1988). Therefore, the user can concentrate on 
the system and the situation at hand without relying on unnecessary and wasted 
cognitive effort. These claims of EID have recently been empirically supported through 
research conducted by Vicente (Vicente, 1991). In the end, the EID framework 
provides an interface which takes advantage of the user's stronger skills and supports 
those that are not as good. 


Ecological Task Analysis 

The ecological approach has also motivated the development of a task analysis 
method to guide display design. A task analysis method predicts the cognitive activity 
required of the user in performing complex tasks. This model of cognitive activity is 
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created by analyzing how the structure of the task environment influences cognitive 
activity. By developing such models of the user and the environment, the design of 
various system components, such as the display interface, can be geared towards 
alleviating some of the cognitive burden demanded by the task, as well as improving 
overall user performance. 

The Ecological Task Analysis frameworic, as proposed by Kirlik (in press), 
begins by describing the structure of the task environment Two different descriptions 
are necessary for this task. The first details the environmental surface structure. This 
structure is a description of the perception and action interface that exists between the 
user and the task at hand. The second examines the environmental depth structure. This 
structure is a description of the covert relationships that link the surface perception and 
action structures. 

The surface structure is described in terms of its perceptual structure and its 
action structure. The surface perceptual structure is comprised of that information 
within the environment which is readily perceivable by the user. This information 
includes, for example, information which is displayed in human-machine systems to 
inform an operator of system states. Information which is present but not easily 
detected by the user is part of the depth structure. The surface action structure can best 
be thought of as those actions made readily available by interface controls. Actions 
which must be performed but cannot be performed readily with interface controls are 
part of the environmental depth structure. 

Performing an ecological task analysis involves determining how the surface 
perceptual structure specifies the surface action structure. When the surface perceptual 
structure specifies the surface action structure, the perceptual guidance of action is 
possible. Two separate models are constructed during this type of analysis. The first 
details the surface perceptual structure by describing what perceptual cues are available 
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to the user. The second nxxlel provides a description of the user's action capabilities. 
An ecological task analysis then simply involves creating these two models and 
identifying where mismatches occur between the environmental representations. 
Mismatches between the surface perceptual structure and the surface action structiu’e 
are indicative of demands for post-perceptual cognitive activity to effectively guide 
action. This process is demonstrated in the following research by performing such a 
task analysis on a particular system. 

Kirlik (in press) emphasizes that in order to perform this type of analysis 
properly, the modeling of the environmental depth structure must occur after that of the 
surface structiuo to ensure that incorrect assumptions are not made concerning what 
cognitive activities are guiding the user’s behavior. This constraint in a sense is what 
makes the ecological task analysis unique. It considers how perception and action 
abilities determine necessary cognitive processes rather than assuming certain cognitive 
processes first and then determining how those processes define the functionality of 
perception and actions. 

There are four possible results of an ecological task analysis process. These 
results have been indicated in Table 2-1. Each type of result indicates a different kind 
of mapping between the surface perceptual structure and the surface action structure of 
the interface. 

The first possible result is perceptual overspec^cation of action. This result 
occurs when many distinct perceptual cues and situations point the user towards 
performing one particular action. This situation poses a problem by forcing the user to 
undergo unnecessary categorical or instance-based learning in order to identify the 
relations between perception and action. By arranging the cues so that perceptually 
salient features are distinguishable in a like manner as the actions they reflect, 
perceptual overspecification can be avoided. 
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Table 2-1. Possible Results Of An Ecological Task Analysis 


Perceptual Many distinct cues point towards perfomiing one 

Overspccification particular action 


Perceptual Not enough perceptual information exists to identify the 

Underspeciflcation proper action to p^orm 


Perceptual Ideal situation: direct relationships exist between 

Specification perceptual cues and specified actions 


Perceptual Perceptual information does not map to any specific 

Misspecification action 
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Perceptual underspecification of action is the second level of agreement that 
may exist between the models. In this case, not enough perceptual information exists to 
identify to the user the proper action to perform. This situation often occurs when the 
user must decide between actions based on what is remembered, not what is presented 
on the display interface. In addition, beside information concerning the past system 
states, the display may not provide cues indicating future states. When these situations 
exist, the user must formulate models of the system in order to successfully interact 
with it. These often take time and much effort to construct. As a result, the skill 
acquired from them only surfaces after considerable learning. Incorporating memory 
aids and/ or predictor displays are possible means of remedying such 
underspecification. 

The third type of agreement is perceptual specification of action. This 
agreement is the ideal situation. Here, direct relationships are fostered between the 
presence of perceptual cues and the specification of actions. As a result, skilled 
performance arises without the user requiring significant learning or other cognitive 
demands. 

The final situation that may occur is perceptual misspecification of action. This 
situation is characteristic of when perceptual information does not map to any specific 
action. In these circumstances, skilled performance is not solely guided by perceptual 
information. Effective performance requires the user to possess some other knowledge 
which allows him/her to overcome perceptual misspecification. To remedy this 
situation, the perceptual information must be altered or replaced with action specific 
information, if possible. 

Simply stated, the goal of the ecological task analysis is to identify what 
perceptual information exists to guide action. Once the levels of agreement existing 
between the perceptual and action environmental models have been determined and any 
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existing problems diagnosed, possible remedies can be suggested and implemented to 
alleviate them and any associated cognitive burden. It is the purpose of this research to 
determine how effectively this goal is accomplished. Does the ecological task analysis 
framework allow the researcher to determine where problems exist in the display 
interface? Will it aid in the modification of the display if necessary? Will these 
modifications actually improve user performance and reduce cognitive burden? These 
questions are examined in the following chapters in an attempt to determine if 
ecological task analysis can indeed serve to improve display interfaces and, thus, user 
performance. 



CHAPTER in 


THE STAR CRUISER TASK 


The effectiveness of Ecological Task Analysis as a display design framework 
was assessed in the context of a laboratory task known as Star Cruiser. This 
simulation was designed by Alex Kirlik of the Georgia Institute of Technology and 
Robert J. Shively of the NASA - Ames Research Center and operates on the Macintosh 
computer. Simply stated, the task involves a user controlling a spaceship through the 
use of a standard Macintosh mouse. The ship, or Star Cruiser, travels to various solar 
systems collecting precious commodities. The user is awarded points for each bit of the 
commodities that are returned to the home port In a sense, especially from the user’s 
viewpoint. Star Cruiser is very much like a video game. From the researcher’s 
perspective, however, this interactive simulation may be used to explore issues 
involving decision making, skill acquisition, and display design. In this particular case, 
the research question being investigated is how effective the ETA framework is in 
display enhancement aiding. What follows is a description of Star Cruiser including 
the various components of the task and the rules which govern the playing of the 
“game.” Also included throughout will be a detailed look at how these elements are 
displayed to the user while performing the task. Photographs of the Star Cruiser 
interfaces have been included in Appendix A. 



Beginning The Task 


To start the simulation once it is ready, the user simply needs to press and 
release the mouse button. The user will then be able to interact with the task until the 
session is over. The session will end if either Star Cruiser explodes (discussed below) 
or if the time allowed expires. The time remaining in the session is displayed in the 
upper right hand comer of the screen. 


Star Cruiser 

Star Cruiser Movements 

Star Cruiser, a yellow and blue object with “NASA” written on its side, is the 
ship which the user manipulates during the task (see Figure A-1). The user directly 
controls the cruiser’s flight speed and direction. This manipulation is accomplished 
when the mouse is used to first place the cursor (an arrow) on the cruiser icon and the 
mouse button is depressed. Without releasing the button, the user may draw a thrust 
string, a straight white line, away from the cruiser by moving the mouse. The further 
the arrow is moved away from the craiser, the longer the string becomes. The length of 
the string indicates the amount of thrust applied to Star Cruiser. This length is, 
however, limited. Therefore, there is a maximum amount of thrust that may be applied 
to the cruiser. If the string length exceeds this limit, it will disappear from the display. 
If the user moves the mouse back towards the cruiser, in a sense shortening the thrust 
string, the white line reappears. The direction the string is drawn is the direction in 
which thrust will be applied. One may think of this action as the string pointing in the 
direction that the user wishes the cruiser to travel in. The thrust magnitude and 
direction as dictated by the thrust string act upon the cruiser when the mouse button is 
released by the user. If the button is released before a thrust string is drawn or when the 



16 


string has been drawn too long and has disappeared from view, then the cruiser will not 
be affected and the user will have to restart the thrusting process. 

At the beginning of the task. Star Cruiser has a full load of fuel. This fuel is 
depleted a certain amount whenever a thrust is applied to the cruiser. The amount of 
fuel used is proportional to the magnitude of the applied thrust - the greater the thrust, 
the more fuel that is used. If the application of thrust uses all the remaining fuel, the 
cruiser will explode and the task session will be over. The cruiser’s current amount of 
fuel is monitored via a “Fuel” gauge. This gauge is located on the upper left side of the 
display. Initially, the gauge is colored blue. As the fuel level on board the cruiser 
begins to decrease, the level of the blue bars in the gauge also decreases. Fuel is fully 
replenished each time Star Cruiser returns to the home port. 

The thrust string, it should be noted, is not the only force which will affect Star 
Cruiser’s speed and direction. The gravitational pull exerted by the suns of the solar 
systems in the galaxy also have an effect This effect will be explained in greater detail 
further on in this chapter. Another important point to note is that if Star Cruiser crashes 
into a sun, the ship will explode and, as with running out of fuel, the session will be 
over. 

Commodities 

In order to be successful at this task, the user must be able to load the cruiser 
with commodities from the different planets and return them to the home port. There 
arc two types of conunodities (see Figure A- 2). The first is “Data." Data is considered 
to be any information or non-physical item that a planet may provide. It is represented 
on the display by a light pink color. The other commodity, “Resources,” is represented 
by a magenta color and may be thought of as minerals, plant life, or any other physical 
object Star Cruiser can only transport a limited amount of data and resources. This 
capacity may be monitored by two bar graphs labeled “Star Cruiser” on the left side of 
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the display. Each graph, one representing the amount of data on board the cruiser, the 
other resources, is filled as more of each kind of commodity is brought aboard. The 
data graph filler is colored light pink and that of the resources is magenta. If the user 
tries to load some amount of data or resources that would exceed the cruiser’s capacity, 
then that amount is lost. Once the user returns the cruiser to the home port, it is 
unloaded and the bar graphs are reset. 


Star Base 

The Ihirpose Of Star Base 

Star Base is the Star Cruiser’s home port (see Figure A-1). Returning the 
cruiser to Star Base results in two significant events. Upon docking at Star Base, the 
cruiser unloads any data and resources it may be carrying. Two gauges, one for each 
type of commodity, display the total amount of data and resources which have been 
unloaded from the cruiser during the entire session. Similar to Star Cruiser’s capacity 
gauges, these are also bar graphs which are filled with the commodity’s corresponding 
color. Labeled “Star Base,” these graphs are located below “Star Cruiser” on the left 
side of the screen. It should be noted that the scale for the “Star Cruiser” gauges is such 
that the gauges are totally full when the cruiser’s storage capacity is exhausted whereas 
the scale for the “Star Base” gauges is such that they are completely full when all data 
and resources in the galaxy have been collected and transferred to Star Base. The 
transfer of commodities is indicated on the display by the cruiser’s capacity gauges 
being cleared and the increase in Star Base’s gauges. Points are awarded for each unit 
of data and resources returned to the base. The total score for the session is displayed in 
the upper right hand comer of the display. 

The other event which occurs upon Star Cruiser’s docking at Star Base is that 
the cruiser’s fuel is fully replenished. This event is indicated by the fuel bar graph 
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becoming fully blue. Once the cruiser leaves Star Base, the fuel will be consumed with 
each thrust as described previously. 

Docking At Star Base 

For the cruiser to dock at Star Base, two important conditions must be 
satisfied. The first is that Star Cruiser must be located near enough to the base for it to 
dock. This condition generally implies that the cruiser must be passing over Star Base, 
or at least crossing over the edge of it The second requirement which must be met is 
that the cruiser must be traveling at a slow enough speed. If Star Cruiser is traveling at 
the proper speed while it passes over Star Base, then it will automatically dock, unload 
its haul and refuel. Star Cruiser will depart from Star Base only when the user has 
applied to it a thrust string of sufficient magnitude. This action is the only one which 
involves the cruiser that the user may perform while it is docked. 

Map Views 

During the task, the user is presented one of two different displays. The first 
display, or view, is of the entire galaxy. This view is the sanne as at the start of the 
simulation and is referred to as the “global” map. The other view displays the contents 
of a particular solar system and can be thought of as a “local” map. 

Global Maps 

The global map shows the entire galaxy including the locations of Star Cruiser, 
Star Base, and the solar systems (see Figure A-1). The Star Base appears as a green 
pentagon with a large white star on it Its position never changes. The solar systems 
are identified by their “suns.” Similar to Star Base, the suns' positions within the galaxy 
are fixed. Each sun conveys two pieces of information. The size of the sun indicates 
the amount of gravitational pull it exerts. The larger sizes exert greater force. As 
previously mentioned, this force will act upon Star Cruiser, thus affecting its speed and 
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direction of flight. The sun’s color informs the user of the amount of commodities 
contained within that solar system. Based on the number of planets in each system, 
those systems which contain the highest percentage of possible commodities are 
assigned the brightest suns. There are five different colors which may be used. They 
range from yellow, the brightest, and increase in redness until the final color of dark 
red. 

Global maps also present the user with one other piece of information. At the 
start of the task, the only clue as to the contents of the solar systems is the color of their 
respective suns. Once a solar system has been visited, though, the amount of data and 
resources contained within it are displayed on the sun in the global map in the form of 
pie graphs. Each “full pie” is divided into halves, one for each type of commodity. The 
left half corresponds to the data content and is colored light pink accordingly. The right 
side is assigned to the resources and is magenta in color. Each half is divided into 
discrete fourths to represent the possible amount of data or resources that the solar 
system may possess. For example, if the system contains anywhere from S 1 % to 75% 
of the possible total amount of data, then three-fourths of the left side pie half will be 
filled in. If on the other hand, if it contains greater than 75% of the total amount, then 
the full half would be colored. If no pie pieces of a particular color are present, then 
that solar system does not contain any of that particular commodity. 

Local Maps 

A local map (see Figure A-2) of a solar system is displayed automatically 
whenever Star Cruiser travels near its respective sun in the global map. (Therefore, the 
ship will only crash into a sun in the local view). The global map automatically returns 
when the cruiser travels beyond the screen boundaries of the local map. When the local 
map appears, the display shows the system’s sun located in the center of the screen. 



20 


Orbiting around the sun arc anywhere from zero to six planets. There are eight possible 
orbits where Orbit #1 is closest to the sun. Each orbit can only have one planet. 

There are two types of planets which may be present. A solar system may 
contain any number (up to a total of six) of each. The two types are identified by their 
colors - blue and green. Blue planets cannot “support life” whereas green planets can. 
The significance of this distinction will be explained later. As with the suns in the 
global map (once a solar system has been visited), the planets also show pie pieces. 
These, however, show the amount of data or resources contained on the one planet. The 
pie charts work the same way for both the color of the pieces and the amount of the 
pieces displayed. 

Similarities B etween Mans 

Regardless of whether the user is viewing a global or local map, there are some 
items which are always displayed. The gauges depicting Star Cruiser capacity. Star 
Base contents, and fuel are always present in both maps. Additionally, the user’s score 
and the time remaining in the session are also visible in either map. The tools (these 
probes, satellites, etc. will be discussed later) available to the user during the task are 
constantly present as well. 


Tools 

To assist the user in performing the task required in Star Cruiser, tools are 
provided. These instruments arc displayed at the top of the screen in both local map 
and global map modes (see Figure A-1 and A-2). There arc five tools with which the 
user can perform several different functions. The tools are, in the order that the are 
displayed on the screen from left to right: probes, satellites, science ships, robot miners, 
and min er ships. All but the first are instrumental in the collection of data and 


resources. 
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CoUection Tools 

To use any of the collection tools (all the tools except for the probes), the user 
must first place Star Cruiser in orbit around a solar system’s sun (see Figure A-3). In 
order to accomplish this task, the cruiser must be present in a local view of some solar 
system. Like the docking process mentioned earlier, two conditions, speed and 
location, must be satisfied for the cruiser to orbit the sun. First, the cruiser must be 
traveling at a slow enough speed so that the “pull of the sun” will automatically place it 
in orbit. Of course, the cruiser will only enter orbit if die second condition, proper 
location, is met. Star Cruiser will orbit around the sun in the ninth orbit Therefore, if it 
is traveling at the proper speed as it intersects the ninth orbit, the cruiser will go into 
orbit Once Star Cruiser is in orbit, it will continually revolve around the sun without 
any assistance from the user. The cruiser will also become highlighted with a green 
outline indicating to the user that it is, in fact in orbit 

Once Star Cruiser is in orbit, the user may deploy any of the collection tools. 
The tools are used to gather the data and resources from the planets and transfer these 
commodities to the cruiser. To deploy a tool, the user must first select it from the top of 
the screen by placing a mouse controlled arrow over the desired icon and clicking the 
mouse button. The tool will change color indicating that it has been selected. The next 
step in deploying the tool is to place the arrow over the orbiting cruiser and press down 
on the mouse button. While still holding the button down, the user can now deploy a 
tool by drawing a deployment string from the cruiser to one of the planets. Unlike a 
thrust string which has limited length, the deployment string has the capability of 
reaching any planet within the solar system. When the arrow is over the intended 
planet, the user must release the button. If the user is successful at “hitting the target,” 
the icon will be seen traveling from the cruiser to the planet where it will be displayed 
next to the lower right-hand comer of the planet The associated tool icon will also 
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disappear from the selection bar at the top of the screen. The user is provided with a 
limited number of tools. If the planet is missed, then the user must simply click on the 
cruiser once more and try hitting the planet again. If, however, the user is unsuccessful 
at clicking on the cruiser, the tool must be selected again in order to deploy it. When 
the cruiser is missed, the tool icon will revert back to its original color to indicate to the 
user that it is no longer selected. 

There are rules which govern the deployment of the collection tools. Each tool 
collects a certain type of commodity, has restrictions on which planets it may be 
deployed to, and behaves differently once it is deployed. 

Satellites . Satellites collect data (light pink) from planets. They may be 
deployed to either the life-supporting (green) planets or those that do not (blue). Once a 
satellite is deployed, it will remain at the planet collecting all possible data until the user 
returns it to the cruiser. 

Science Ships . Like satellites, science ships collect data. Whereas the 
satellites are strictly mechanical in nature, the science ships may be thought of as being 
manned ships. Therefore, they may only be deployed to the life-supporting green 
planets. Another difference between the satellites and the science ships is that after the 
science ship is finished collecting data from one green planet, it will seek out another 
which also contains data. If one is found, the ship will automatically “jump” to that 
planet and begin adding its data to what was already collected. If no other data-rich 
planets are present, the tool will remain at the last planet it collected data from until the 
user returns it to the cruiser. 

Robot Miners . Robot miners collect resources (magenta) from the planets. 
With this exception, they behave similarly to the satellites; they may be deployed to 
either type of planet and they will only collect from one planet 
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Miner Ships . Miner ships are manned ships which collect resources. Since 
they arc manned, they, like the science ships, may only be deployed to the green 
planets. A miner ship will jump, in the same fashion as a science ship, from green 
planet to green planet continually collecting resources until no others exist or until the 
user returns it to Star Cruiser. 

The user is not limited in the number of tools deployed in any one solar 
system. In addition, more than one tool can be deployed to the same planet For 
example, two satellites can be deployed to the same planet a satellite and a science ship 
can both be sent to a green planet, or a robot miner and a satellite can both collect 
commodities from the same planet. If multiple tools are deployed to the same planet 
they will all be displayed in the same position next to the planet thus overlapping one 
another. The displayed icons flicker, however, letting the user know that more than one 
tool is located there. In addition, if multiple tools of the same type have been deployed 
to the same planet they all will collect the appropriate commodity. Once a tool has 
been deployed to a planet it will automatically start the collection process. The user 
can watch as the pie pieces representing the data or resources on that planet begin to 
disappear. This disappearance indicates to the user that the tools located at that planet 
are indeed collecting the data and/or resources. The same holds true for the pie pieces 
displayed on the suns in the global map. The absence of a pie piece indicates that all of 
that type of commodity has been collected. 

The collection tools will continue to collect as much data or resources as 
possible unless the user intercedes first At any time, right after deployment, in the 
middle of the collection process, or once all possible data or resources have been 
collected, the user may retrieve the tool back to Star Cruiser. Retrieving a collection 
tool transfers any data or resources that it may have collected from the planet(s) to the 
cruiser. In order to retrieve a collection tool. Star Cruiser must again be in orbit. If it is 
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in orbit in a solar system where tools have been deployed, the user needs only to click 
on the collection tool desired for retrieval without releasing the mouse button. A 
retrieval string can then be dragged to the cruiser where the button is then released. 
(Note that this process is merely the deployment process in reverse.) If done properly, 
the user will witness the tool returning to the cruiser. It will then take its original place 
back at the top of the screen and, if anything has been collected, the appropriate Star 
Cruiser gauge will increase. The tool may then be deployed once again. If multiple 
tools have been deployed to a single planet, their order of retrieval will be the same as 
their displayed order at the top of the screen from left to right (satellites, science ships, 
robot miners, miner ships). As mentioned previously. Star Craiser has a limited storage 
capacity. If a retrieved tool contains more data or resources than the cruiser can hold, 
all of that gathered commodity is lost In addition, the user loses the service of that tool 
for the remainder of the session. This loss is indicated by the appearance of the tool 
icon at the top of the screen with a yellow ‘X’ over it 

The user is not required to wait for the collection tools to finish gathering the 
data and/or resources before doing anything else. If so desired, the user may visit other 
solar systems to either begin collecting commodities or to gather what has already been 
collected. If Star Cruiser’s capacity is near full, then the user may want to dock the 
cruiser at Star Base in order that more data and resources may be collected. What the 
user decides to do is dependent upon his or her particular method for performing the 
ta sk . It is important to rcmembcr, however, that it is permissible for the user to leave 
the solar system after deploying tools. Star Cruiser, though, must first be taken out of 
orbit. This task is accomplished by applying thrust to the cruiser. If its magnitude is 
great enough. Star Cruiser will break out of orbit and head in whatever direction the 
string was drawn. The thrust string may only be drawn if none of the tools have been 
selected for deployment Otherwise, a deployment string will be drawn. Similarly, if a 
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tool is not selected, any strings drawn from the cruiser will function as a thrust. The 
user always has the option of having Star Cruiser return to a previously visited solar 
system since the deploying and retrieving of collection tools can only occur if the 
cruiser is in orbit in the system of concern. 

£ml2CS 

Probes are the rocket-like icons displayed first in the tool selection bar at the 
top of the screen. Unlike the other tools, they are not used for gathering data or 
resources. Instead, their function is to transmit information about the solar systems 
back to Star Cruiser. The probes’ method of deployment is slightly different as well. 
The key variation is that the cruiser does not need to be in orbit for a probe deployment 
to occur. The user may perform the action in either the global or local maps. Like the 
collection tools, the user must first select a probe for deployment from the top of the 
screen. Then, after clicking on the cruiser, a deployment string is dragged to the sun of 
the solar system in which the probe is to be sent, rather than to a planet If the current 
display is the global map, the user will see the probe icon move to the selected sun 
where it will be displayed next to it the lower right-hand comer. This indicates that a 
probe has been deployed to that solar system. If the local map is displayed, the probe 
will be seen orbiting the solar system’s sun in the ninth orbit ring. (Note: The users are 
not informed of this fact since this orbit is the same as Star Cruiser’s orbit. If told, the 
users would be more likely to use the probe to locate the ninth orbit for the purpose of 
placing the cruiser into orbit. This “trick” was left to the user to discover.) The process 
of retrieving a probe is the same as for the other tools: the cruiser must be in orbit in 
the proper solar system. 

Deploying a probe to a solar system has the same effect as visiting it with the 
cruiser. Upon arriving at a system, the probe reveals the amount of data and resources 
contained within it by displaying the pie pieces on the sun in the global map. In 
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addition, the user now has the means to “look into” the system without having to 
actually visit it with the cruiser. When viewing the global map, if the user clicks on the 
sun of a solar system where a probe has been deployed, the local map of that system 
will be displayed. The user, however, will not be able to perform any actions within 
that view since the cruiser is not present Clicking on the sun in the local map will 
return the display to the galaxy view. This process will also work for those solar 
systems that were previously visited by the cruiser, even if a probe has not been 
deployed to them. Basically, this “looking into the solar system” may be performed on 
any suns which have been visited or probed at any time. 

Switching Between Maps 

The user has the ability to switch to a local map from the global as Star Cruiser 
travels through the galaxy. The user at times may also switch from a local to a global 
map. In order to do so, the cruiser must be in orbit in some solar system. By clicking 
on the sun in a local map, the display will switch to the global map. The Star Cruiser 
icon will be displayed next to the sun of the solar system where it is currently in orbit. 
Again, since the cruiser is not actually located in the global map, no other actions, with 
the exception of switching between maps, can be performed. After switching to the 
global imq), the user has two options. One is that the user may switch back to the local 
mt^ of the solar system which contains Star Cruiser by clicking on the appropriate sun 
in the galaxy view. The other choice is that, if other solar systems have been previously 
visited or probed, the user can view one of them by clicking on their corresponding sun. 
Clicking on the sun in the local view will return the display to the global map. This 
ability to switch between maps is useful in identifying the contents of various solar 
systems without having to direct Star Cruiser to actually visit them. The user must 
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remember though, that other actions may only be performed when operating in the same 
view as Star Cruiser. 


Scenario Files 

Scenario files act as an initialization file which creates the galaxy. The number 
of solar systems in the galaxy, the number and types of planets in each system, the 
annount of data and resources on each planet, the amount of each tool initially available 
to the user, and the locations of the solar systems and Star Base can all be manipulated 
by the researcher through the scenario files. The files also allow certain task parameters 
to be varied. Examples of these include the number of points awarded for the data and 
resources, the fuel consumption rate, and the time given to perform the session. 
Appendix B lists all parameters that may be adjusted in the scenario files. 



CHAPTER IV 


AN ECOLOGICAL TASK ANALYSIS OF THE STAR CRUISER TASK 


An ecological task analysis was performed on the Star Cruiser simulation. 
Three experienced users were created by having them continually practice the game 
until no further improvements in their overall score were detected and, thus, had 
reached a maximum plateau (this occurred after approximately 20 sessions). These 
users were then videotaped while playing Star Cruiser. Verbal protocols and written 
questionnaires were also given to the subjects. Reviewing these three items for each 
subject revealed the methods for performing the task that the experienced operators 
employed. As a result, the Star Cruiser simulation was broken down into the 
following tasks: 

1. Deploy Collection Tool 

2. Retrieve Collection Tool 

3. Deploy Probe 

4. Retrieve Probe 

5. Place Star Cruiser Into Orbit 

6. Remove Star Cruiser From Orbit 

7. Dock Star Cruiser At Star Base 

8. Release Star Cruiser From Star Base 

9. Change View To Galaxy/Solar System Map 

Each task was analyzed and described in terms of whether or not perceptual cues 
existed to indicate the availability of the actions comprising the task. 
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Deploy Collecrion Tool 

Only under certain conditions may a user deploy one of the four collection 
tools: satellites, robot miners, science ships, and miner ships. Star Cruiser must be in 
orbit around a sun which also has orbiting planets. In addition, the user must be 
viewing the local map of that system. If the user wishes to deploy either a science ship 
or a miner ship, then planets which support life, green planets, must also be present in 
the solar system. 

Each subject possessed a different method for determining where and when to 
deploy collection tools. One subject, for instance, would first send out science ships 
and miner ships whenever possible before satellites and robot miners. Her method was 
to collect as many commodities as possible as quickly as possible. She viewed the 
collection of data and resources using the science ships and miner ships to be quicker 
since she would not be required to make as many deployments. She, more than any 
other subject, appeared to have the greatest difficulty with performing the necessary 
tracking task required to deploy the collection tools. 

Another subject would normally deploy only satellites and robot miners since 
these did not move from planet to planet and she was thus better able to remember how 
much data or resources each tool had collected. Only if one green planet was present 
would she then send out a science ship or miner ship. This means of operating the task 
was developed in order to reduce the risk of a science ship or miner ship collecting to 
much data or resources and thus overloading the cruiser when it is retrieved. The 
subject would also deploy as many tools as she had available, even if the cruiser would 
not be able to cany all of the gathered data and/or resources. It was her reasoning that 
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she could always return to the system after unloading Star Cruiser at Star Base and 
renieve the remaining tools. 

A third subject had yet a different method for deploying collection tools. He, 
like the first subject, would deploy science ships and miner ships first. Unlike the first 
subject though, this subject would send out multiple science ships and miner ships to 
speed up the collection process even more. This subject differed from the other two in 
that, after retrieving the other tools, he would deploy satellites and robot miners to other 
planets to collect a fraction of their data and/or resources. His method for this task 
focused on returning to Star Base with Star Cruiser as fully loaded as possible whereas 
the other subjects were content with the cruiser not being as full. He usually did not 
leave tools in the solar system once he had the cruiser leave. There did seem to be one 
bit of methodology that all subjects had in common though. In deciding where to send 
a collection tool, the subjects generally did not send more than one data collecting tool 
(satellite or science ship) or resource collecting tool (robot miner or miner ship) to the 
same planet. 

The fact that all subjeas had differing methods for performing this task serves 
as evidence to the lack of perceptual cues that exist for aiding the users in deciding 
when and where to deploy the collection tools. If these cues were present to indicate 
when to perform the action, then there would most likely be more similarities between 
each subject's method- This theory is illustrated by the subjects' common attitude that if 
a tool is already collecting one type of commodity from the planet, then no other tool 
which collects a similar commodity should be deployed to that planet A cue does exist 
for this situation even though it is informing the subjects not to perform the action 
rather then to do so. When a collection tool is deployed to a planet, it is displayed next 
to it Therefore, the user can see which tools have been sent to which planets. As a 
result, the user knows that the deployment of any other tool to that planet (which 
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collects similar commodities) is essentially pointless since the tool presently there will 
collect all the data or resources that the planet contains. The only benefit that deploying 
another tool which collects the same type of commodity to that planet would possibly 
serve is that the tools would gather equal amounts of the available commodity. As a 
result, one tool will be less likely to transport too much of the commodity back to the 
cruiser, and thus overload it, once it is retrieved. This benefit never seemed apparent to 
the subjects since they were not observed performing an action of this type. The 
absence of this type of action is mostly likely due to the lack of cues that would inform 
them of how much each tool has collected, let alone whether this action was even 
possible. 

None of the subjects appeared to have difficulty determining when the 
deployment of tools could be performed. The subjects were instructed during their 
training about the conditions that must be met to perform this action. With this 
knowledge, they were able to recognize easily when the conditions were satisfied. 
Perceptual cues, such as Star Cruiser being highlighted upon reaching orbit, the absence 
of the cruiser and/or the planets when the subjects were looking at a view of the galaxy 
or another solar system, and whether any planets were present in that solar system at all, 
also provided the necessary support in determining when the action could or could not 
be performed. The fact that the tools were always present at the top of the screen, thus 
appearing as if they could be selected even when they could not, did not cause the 
subjects any difficulties. If, for some reason, they tried to select one of the tools when 
that action could not be performed, they would soon discover that their attempt was not 
allowed due to their failure to do so. Overall, the cues were successful at specifying the 
availability of the action. 

There was, however, one area of underspecification. Though they knew they 
could perform some type of deployment, the subjects had problems determining which 
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tools should be used to collect which commodity and which type of planets they could 
be sent to. Most of the subjects apparently were successful at memorizing the 
differences between the tools. There was one subject though, that had difficulty 
throughout the sessions. The tools themselves provide no cues as to what they do. 

They are not coded in any fashion whether it be, for example, color, size, or location (at 
the top of the screen where they may be selected). Some form of cue should be 
incorporated into the display to assist the user. Successful implementation of such cues 
should result in the user's reduced confusion concerning each type of tool's functions 
and constraints as well as reducing their need to rely on memory to determine them. 

Deploying collection tools was consistently regarded as one of the more 
difficult actions to perform. Though much of this difficulty was attributed to the 
tracking task involved, some of the blame can also be placed on the amount of mental 
effort the subjects had to use to accomplish the action. This reasoning is suggested by 
the subjects considering this action to be slightly more difficult than that of retrieving a 
tool which essentially incorporates the same type of tracking task. The subjects had to 
remember to which type of planet each tool could be sent as well as what each 
collected. Even more difficult though, they had to determine exactly to which planet to 
send a tool. This decision often required the subject to consider the amount of data or 
resources available on that planet (indicated by the "pie pieces" displayed on each), the 
amount currently on board Star Cruiser, the amount of data or resources being currently 
collected by other tools, whether the deployed tool would move from planet to planet, 
and how much time was left in the mission. Though much, but not all, of this 
information was presented to the subjects in one form or another, they were required to 
interpret each bit and relate each piece of information to the odiers in order to make a 
decision. This process was often quite complex and thus it is understandable why this 
action was considered to be so difficult. Ensuring the proper presentation of 
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infonnation by the current perceptual cues, along with the introduction of new cues, 
should help to make this action an easier one for the users to perform. 

Retrieve Collection Tool 

In order to retrieve collection tools, similar criteria to that for deploying the 
tools must be satisfied. This criteria includes viewing a solar system where Star Cruiser 
is in orbit. The only difference in the criteria is that, in that solar system, tools deployed 
earlier must still be located at the planets. 

Subjects generally used criteria similar to each other in determining when it 
was, and was not, appropriate to retrieve a collection tool. The subjects would retrieve 
a tool if two conditions were satisfied. The first is that the tool had finished collecting 
all of the data or resources that were available. The other was that there was room 
aboard the cruiser to carry the collected commodities. The remaining time left in the 
mission also played a role in determining if a tool should be retrieved or abandoned so 
that the cruiser could return to Star Base with what was already on board. 

The subjects had very little difficulty in determining when they could perform 
this action. Cues were present and noticeable to indicate when a tool could be retrieved. 
As in deployment, the highlighting of Star Cruiser once it is in orbit informs the user 
that that portion of the criteria has been satisfied. In addition, the subjects knew which 
tools could be retrieved due to them being displayed next to a planet in the solar system. 
Confusion does exist, however, in determining which tool will be retrieved first when 
multiple tools are present at the same planet. Since their displayed icons overlap, there 
is no apparent cue to indicate which tool will be retrieved first when the user selects 
one. In order to overcome this lack of infonnation, the subjects were instructed that the 
order of retrieval was identical to the order, ftom left to right, of collection tools 
displayed at the top of the screen. It was observed though, that even with this 
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knowledge, the subjects would attempt to retrieve one tool from a planet containing 
multiples, and inadvertently retrieve the wrong one. As a result, they would usually 
retrieve the correct tool and then deploy the one that was incorrectly brought back in 
order to finish collecting the available commodity at that planet. This problem could be 
avoided by simply locating each type of tool in a different position around the planets. 
With this exception, the availability of this action was generally well perceived by the 
subjects from the existing perceptual cues. 

The ease in deciding whether or not to perform this action was variable 
throughout the mission. It greatly depended on the second of the two criteria mentioned 
earlier - how much data or resources Star Cruiser currently had on board. If the cruiser 
was empty, then sufficient perceptual cues existed to suggest to the user that a tool 
should be retrieved. The fact that the tool had completed collecting all possible 
comrrKxlities would be indicated by the disappearance of the "pie pieces" which show 
how much data or resources is present on the planet Since the gauges which convey 
how much collected commodities Star Cruiser contains would be empty, the user would 
know that retrieving a tool was more than likely an appropriate action to perform. As 
Star Cruiser contained more and more corrarrodities though, the decision to retrieve a 
tool became more and more difficult Even though the user could still readily determine 
if the tool had completed its collection task, insufficient cues existed to inform the user 
whether or not the comriKxiities collected could be safely loaded onto the cruiser. The 
problem is that no direct relationship exists between how much of the commodities, 
represented by the pie pieces, is collected and how much the gauges indicating the 
cruiser's load will increase once those tools arc retrieved. 

Unless it was learned exactly how many pie pieces completely loaded the 
cruiser (which two subjects did learn over twenty sessions according to discussions with 
them), the user would never be quite sure how much additional data or resources can be 
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brought onto the cruiser before it exceeded capacity. Even when it was learned that, for 
example, three full planets completely loaded the cruiser, difficulties still arose when 
the subjects were forced to deal with planets which contain only a fraction of their total 
capacity of commodities. This problem was compounded further by the pie pieces for 
two reasons. One is that the same size pie piece represented a range of amount of data 
or resources. For example, a half of a pie piece (one full pie piece equals one half the 
size of the planet) of data could represent between one-fourth and one- half of the 
planets capacity for a type of commodity. There is no way to determine the exact 
amount Contributing even more to the problem is that the gauges indicating the 
cruiser's current load merely present qualitative information - how full is Star Cruiser. 
As a result it is also difficult to determine exaedy how much data or resources the 
cruiser already contains as well as how much the gauges will increase by collecting a 
certain size pie piece. The second reason is that the pieces will disappear as the 
connmodity is collected. Though this disappearance serves as a good cue indicating 
when the collection process is complete, unless the user remembers how big of a pie 
piece was present before the tool was deployed to the planet there is no way for the 
user to know how much data or resources the tool has collected. This fact is especially 
true for science ships and miner ships which can visit multiple planets before being 
retrieved. This situation is an even greater problem when the user has done something 
(i.e., returned the cruiser to Star Base) which results in the viewing of the galaxy or 
another solar system. 

The goal of the user is to collect as many commodities as possible and return 
them to Star Base. As a result, this action of retrieving collection tools is one of the 
most crucial. The users' success at completing their mission ultimately depends on their 
ability to make good decisions concerning when to retrieve a collection tool. The 
display should be designed to aid the decision-making process, not hinder it. Therefore, 
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enhancements should be made to the current display in order to improve the users' 
chances of success. Creating gauges and pic pieces which have direct relationships to 
each other is one such improvement. Others may include memory aids which will help 
the users remember exactly how many commodities have been collected by a tool. 
Warnings can also be incorporated indicating when Star Cruiser has reached near 
capacity or even when the retrieval of a particular tool will overload it. These are just 
some of the possible enhancements which can assist the users in deciding when to 
retrieve a tool. As a result, not only should the users' performance of this action 
improve, but so should that of their overall mission as well. 

Deploy Probe 

A probe may be deployed at any time except fw when Star Cruiser is docked 
at Star Base. Deploying probes has no effect on points or fuel consumption and can 
therefore be done without great effect on the system states. Subjects used the probes to 
obtain information concerning ninth orbits and solar systems’ supply of commodities. 

When determining to which solar system to send the cruiser, subjects would 
dispatch probes to the systems, usually those with the brightest suns, under 
consideration. They would then be allowed to view the planets in those systems and 
ascertain the amount of conmodities available for collection in each. The only benefit 
this feature served was to help choose which system to go to amongst those with the 
brightest suns. However, other factors often contributed to this decision as well. 
Proximity to Star Cruiser, the closeness of neighboring solar systems (subjects' methods 
sometimes involve collecting data from a grouping of solar systems), and the remaining 
rinv» in the session often served as determinants in deciding which solar system to enter 
first 



As mentioned, probes were also used to locate the ninth orbit around a sun. 
Subjects used their knowledge of the probe's orbit path to help locate the proper orbit 
for Star Cruiser. This use of the probe was not necessary at all times since other cues 
exist to help find the proper orbit (refer to Move Star Cruiser into Orbit) . 

These reasons for deploying probes are driven by the subjects’ desire to obtain 
additional information about the galaxy. There are no cues that the subjects perceive 
which cause them to perform this action. In other words, the need to perform this 
action is only inferred by the subjects, no information is given instructing them to do so. 
Perceptual cues, however, currently exist to provide all the same information to the 
subjects (i.e., sun color, planets orbiting near the ninth orbit). As previously stated, the 
only time that probes may not be deployed is when Star Cruiser is docked at Star Base. 
There are no perceptual cues, though, that inform the subjects of this constraint. Only 
their inability to do so suggests to the subjects that a probe cannot be deployed at that 
time. Also, the only cue that exists to indicate that a probe has been selected for 
deployment is when the user has highlighted one at the top of the screen. This lack of 
cues poses a problem, however, when Star Craiser enters a solar system while a subject 
is attempting to perform this action. On two separate occasions, with two different 
subjects, this problem resulted in Star Cruiser crashing into the sun. The subjects 
selected a probe to deploy while Star Cruiser was in the galaxy view. During this 
process, the cruiser drifted into a solar system. The subjects' initial, and only, reactions 
upon seeing Star Cruiser drifting towards the sun was to apply a thrust to the cruiser 
away from the sun. The probe, however, was still selected for deployment. Therefore, 
the only actions the subjects could do successfully would be either to pull a string from 
Star Craiser to the sun to deploy the probe tw to unselect the probe. Since the subjects 
were concerned solely with applying a thrust to the craiser away from the sun, they did 
not realize that they had to perform one of the other actions first As a result, the thrust 



38 


was not applied to Star Cruiser and it crashed into the sun. Though their inability to 
perform the desired thrust action serves as a cue, it is embedded too deeply within the 
structure of the interface to be of use. The subjects simply feel that the applied thrust 
was not great enough to overcome the sun's gravitational pull and they continually try 
the action again. Some other form of cue is required in this situation to indicate to the 
subjects that they are performing an action unsuccessfully and that they should attempt 
another. The cue must serve to enlighten the subjects that their intended action is one 
that is not readily available. 

Overall, there is no special need to use perceptual cues to inform the user when 
to deploy a probe. Most information that can be gained by doing so is present at all 
times. Cues that exist, though, at the depth structure level, especially those which 
indicate that the user needs to unselect a probe to perform a thrust, need to be brought to 
the surface. In addition, perceptual cues should indicate to the user when it is possible 
to deploy probes. 


Retrieve Probe 

A probe may only be retrieved when Star Cruiser is in orbit in the same solar 
system as the probe and the current view is of that system. If these conditions are met, 
then the user may select the probe and draw a line back to Star Cruiser to retrieve it. 
Retrieving a probe has no effect on fuel consumption or points and can be time 
consuming. The only benefit is that the user now has an additional probe which may be 
deployed. Therefore, if the user determines that enough probes are present on board 
Star Cruiser to con^lete the session, then there is no reason why a probe should be 
retrieved at any time. This observation agrees with the methods used by most of the 
experienced subjects. One subject would, however, retrieve a probe while Star Cruiser 
was waiting for the collection tools to finish gathering commodities from the planets. 
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His reasoning was that since Star Cruiser has visited the solar system, the amount of 
commodities present will always be displayed on the sun in the galaxy view. Therefore, 
depending on the availability of other probes fw deployment, he may retrieve the probe 
while waiting for the collection tools to finish gathering commodities. His decision to 
retrieve the probe was also dependent on his confidence in being able to locate the ninth 
orbit if he ever returned to the same solar system in the future. Of course, if all 
commodities have been gathered from the planets in that system, then there will be no 
need to ever return. 

Though there are no perceptual cues on the display which indicate that this 
action should be performed, the user may determine to select this action based on the 
number of probes displayed along the top of the screen. Whether or not a user will 
retrieve a probe will ultimately depend on whether or not it is felt that it will be used 
a gain. Currently, only the user’s determination of certain criteria, the time remaining 
and where the cruiser should be sent in that time, will contribute to the decision to 
perform this action. As with the subject discussed previously, there are those who do 
always retrieve the probes though, if, for nothing else, to practice the task of retrieving 
an object Because of this action's insignificance to the overall mission goal (collect 
conraxxiities and return them to Star Base), it is probably unnecessary to incorporate 
cues which directly inform the user that a probe should be retrieved. 

Perceptual cues which infram the user when this action can be performed are 
also lacking on the display. Though subjects had very little difficulty with determining 
when a probe could be retrieved, there does exist the potential for some confusion. 
Since a probe ma y be deployed at almost any time, the risk exists that a user may think 
that it can be retrieved at almost any time. This line of reasoning is further supported 
by the fact that in the galaxy view, the deployed probes are pictured next to their 
corresponding solar systems. Though this bit of information may be helpful in 
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determining where the probes are located, it can also cause the user to elect to perform 
this task as a result of the misleading cue. To avoid the misspecificadon between the 
cue (displayed probes) and a possible action it may indicate (can retrieve the probes), 
the cue should either be altered or eliminated regardless of the fact that the user's 
inability to retrieve a probe may indicate the use of an improper retrieval method. The 
key here is not to correct the user once the mistake is made, but to prevent the mistake 
from being made in the first place. 

A user may relate the retrieval of a probe to that of a collection tool. This 
reasoning would then serve as a cue to the user since a tool may only be retrieved 
while Star Cruiser is in orbit, the sanw belief may be had for the probes. Though this 
belief would be correct, the users should not be subjected to the burden of learning it for 
themselves. This similarity between retrieving the tools and the probes should be 
evident from the display itself. Taking all into consideration, for the display to contain 
proper cues, it must not only inform the user of when the probes can be retrieved, but it 
must not mislead the user into thinking they can be when they truly cannot 

Place Star Cruiser Into Orbit 

Star Cruiser will achieve orbit around a sun if it passes through the sun's ninth 
orbit at a slow enough speed. Thus, the user must place the cruiser in the solar system 
view which contains the sun to be orbited. 

Subjects shared a similar reason for wanting Star Cruiser to obtain orbit: to 
deploy collection tools. Unless they had taken the cruiser out of orbit while tools were 
collecting commodities, the subjects generally did not try to place the cruiser into orbit 
for the sole purpose of retrieving collection tools or probes. If a subject made the 
decision to retrieve a probe, it was only while Star Cruiser was already in orbit. 
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Only previous instruction lets the user know that orbit must be obtained. Since 
it is known that Star Cruiser must be in orbit in order to perform any actions regarding 
the collection tools, once the decision has been made to deploy or retrieve a tool, the 
user knows that the cruiser must be placed in orbit. Thus, in a sense, those cues which 
aid the user in deciding whether or not to deploy or retrieve tools (i.e., presence of pie 
pieces on planets; absence of planets) also serve as cues to put the cruiser into orbit. 
There is, however, no direct mapping between the desired action of deploying or 
retrieving a tool and the necessary means for doing so such as first obtaining orbit As a 
result, those users who do not receive instructions prior to attempting a mission may try 
to deploy a tool while the cruiser is not in orbit Of course, their failure to complete this 
action will indicate to them that something is wrong, but they would most likely be 
unable to determine what 

Assuming that the user knows that Star Cruiser must be in orbit to perform 
actions on the collection tools, very few perceptual cues are required to indicate the 
availability of the action. Knowing that the ninth orbit must be located, the user 
generally realizes that only in a view of a solar system may this action be attempted. 
Thus, once the cruiser moves into a view of a solar system, no other requirements must 
be met in order to attempt this action. 

The difficulty concerning this action, according to the subjects, was not in 
determining when to perform the action nor if the action could be performed. It was in 
performing the action itself that presented the most trouble. The subjects often 
complained about how hard it was to locate the ninth orbit, let alone get Star Cruiser 
nnoving at the proper unknown speed so it would automatically place itself into the orbit 
around the sun. As a result, they were often required to continually adjust the cruiser’s 
direction and speed until the cruiser reached orbit. This process proved to be a 
frustrating one which often wasted valuable time. Regarding the speed, there are no 
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cues whatsoever which would indicate to the user that the cruiser has the proper 
velocity for obtaining orbit Only upon seeing the cruiser obtain orbit (Star Cruiser is 
highlighted) will the user learn what the proper speed is. The user, however, usually 
has difficulty remembering what the speed was, or duplicating it This difficulty is 
because the user is required to match it to a pictorial representation of the cruiser’s 
movements. No quantitative information is provided. There are several cues, though 
not intuitive, which may be utilized in locating the ninth orbit around a sun. The first is 
to use the planets as a guide. Eventually a subject will learn that the furthest orbit for a 
planet is one less then that of the cruiser. This bit of knowledge allows the user to 
approximate the ninth orbit’s location based on the orbits of the existing planets. 
Another cue that may be learned over time is that the orbit is elliptical in shape. The 
user can then "picture" the path that the orbit takes at a certain distance from the sun 
and attempt to have the cruiser intercept that pictured path. A third method for finding 
the orbit, one consistently performed by several of the subjects, is to dispatch a probe in 
the solar system where the cruiser is to obtain orbit Learned through practice, the users 
soon realize that the probe travels around the sun in the ninth orbit This "trick" allows 
the user to deploy a probe to identify the orbit which can then be easily located while 
attempting to place the cruiser. Though cues do exist to show where the orbit is 
located, apparently they are not readily perceived by the user unless their existence has 
been learned. 

While the number of cues indicating that this action should be performed, and 
that it can be performed, might be sufficient, additional cues should be added to assist 
the user in placing Star Cruiser into orbit. To be effective, the cues would need to alert 
the user to the location of the ninth orbit This alert can simply be accomplished by 
highlighting the orbit in some manner. In addition, some form of a cue should be used 
to indicate when the cruiser is at the required speed for obtaining orbit This cue may 
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include, for example, the use of a "speedometer" or changing the color of the cruiser. 
Without these additions, users will face difficulties as they attempt to place Star Cruiser 
into orbit. 


Remove S tar Cruiser From Orbit 

This action may only be performed if Star Cruiser is, obviously, in orbit in 
some solar system and the user is viewing that system. Subjects usually took the cruiser 
out of orbit only if one of two conditions was met. If all possible commodities had been 
collected in the system, and all tools had been retrieved, then the subjects removed the 
cruiser from orbit in order to send it to another solar system or to Star Base. On the 
other hand, if more commodities could still be collected but there was no additional 
storage space on board the cruiser, then Star Cruiser was taken out of orbit as well. In 
this case, the cruiser was returned directly to Star Base. 

The number of perceptual cues which would lead the user to perform this 
action are minimal . When presented with a view of a solar system which is void of 
tools, probes, and commodities, the user should realize that there is no reason to have 
the cruiser in orbit. Other cues may also be present which may lead to the decision to 
remove Star Cruiser from orbit These, however, do so indirectly since they actually 
infemn the user that another action should be performed (refer to Dock Star Cmiser at 
Star Base) . In order to perform this action, though, the user must first remove the 
cruiser from orbit . In this case, this cue is satisfactory since the relationship between 
the two actions (one must precede the other) is an easy one for the user to perceive. The 
action of removing the cruiser from orbit is easily triggered by the cues which currently 
exist and, thus, no modifications are necessarily required. 

As mentioned before, the user must be viewing the solar system which 
contains the orbiting Star Cruiser in order to perfonn this action. If not, then the user's 
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inability to successfully remove the cruiser from orbit serves as a cue that the action 
cannot be performed. Otherwise, there are no other cues which indicate this situation to 
the user, nor that the user can perform the action. It was noticed, though, that subjects 
very rarely, if ever, attempted to remove Star Cruiser from orbit when it was impossible 
to do so. In addition, whenever they decided to perform the action, they were in a 
situation which permitted the action. Thus, it appears that whenever the user has 
decided to remove the cruiser from orbit, no additional cues are required to show the 
availability of that action. 

Problems do arise, however, when die user docs not wish to remove the cruiser 
from orbit, but does so accidentally. Unintentionally removing the cruiser from orbit 
may occur as a result of the aforementioned lack of cues which indicate when this 
action may be performed. Several attempts made by the subjects to deploy tools 
resulted in their accidental removing of Star Cruiser from orbit When deploying a tool, 
after selecting it from the top of the screen, the user is required to select Star Cruiser as 
it is traveling around the sun. If the user is unable to track and select the cruiser 
properly, then the tool is unselected. The subjects would not realize this occurrence and 
attempt to select the cruiser again. If successful at the second attempt, since the tool is 
no longer selected, they would actually be placing a thrust on the cruiser. This thrust 
more than likely pulled the cruiser out of orbit at an undesirable time. This error was 
often experienced during the earlier sessions. Even with mme practice, the mistake was 
still made, though not as frequently. The subjects learned that they needed to select the 
tool again before selecting the cruiser. Some form of cue should be present, however, 
to indicate to the user when a string from the cruiser will deploy a tool or probe or when 
it will act as a thrust In other words, a perceptual cue should be implemented which 
will inform the user of the availability of this action. 
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Dock Star Cruiser At Star Base 

Star Cruiser can be docked at Star Base if it is moving around in the galaxy 
and the user is viewing it. The cruiser must be traveling at a slow enough speed as it 
passes over the base in order for it to dock. 

Subjects shared similar lines of reasoning in deciding when to dock the cruiser 
at Star Base. Any one of three situations would result in the subjects abandoning their 
current activities and performing the necessary actions that would lead to the cruiser's 
docking. The main reason people docked the cruiser was when they wished to unload 
its contents. Some subjects attempted to fill the cruiser as full of commodities as a solar 
system's planets would allow, while others were satisfied with partial fillings. Another 
factor which often led to subjects moving the cruiser to Star Base was the time 
remaining in the session. If the cruiser had some commodities on board and time was 
nearly expired, the subjects would dock the cruiser as quickly as possible in order to 
increase their point total. The third factor which compelled people to perform this 
action was the amount of fuel the cruiser had remaining. Lack of fuel would force 
subjects to return the cruiser back to Star Base in order to refuel. Essentially, as long as 
the cruiser had enough cargo space and fuel, the subjects were content with keeping it 
moving through the galaxy and collecting commodities from the various solar systems. 

The availability of this action was usually not questioned by the subjects. 
Though not often attempted, an inability to maneuver Star Cruiser while it is located 
somewhere other than the current view provides the necessary cue to the user that the 
action cannot be performed. Due to the nature of this action though, this cue is not 
often enough. In order to dock the cruiser at the base, the user may have to, or wish to, 
perform other actions first. For instance, if Star Cruiser is in orbit, the user must take it 
out of orbit before the cruiser can be docked at Star Base. Therefore, in some cases, the 
availability of this action depends on that of others and thus the cues which signify the 
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availability of those other actions become significant. The subjects did not seem to 
have trouble with this aspect of the docking action. They took into account such 
relationships between actions and performed them accordingly. 

From subjects' performance, there appeared to be several perceptual cues 
which led them to perform this action, especially when considering the cargo space and 
time factors. The gauges depicting the amount of cargo space aboard Star Cruiser were 
often referred to during the collection of commodities. As a result, the status of the 
cruiser’s cargo space was often known. Thus, if the user deemed it necessary. Star 
Craiser would be returned to Star Base. The presence of the timer during the mission 
provided the subjects with the necessary information concerning the time remaining. It 
too was often checked to determine if Star Cmiser should be returned to Star Base, or if 
any other actions should even be attempted. 

The subjects often ran into difficulty when it came to the third factor. Star 
Cruiser's fuel level was very rarely monitored by die subjects. As a result, the cruiser 
would explode, thus ending the mission, much to the surprise of the user. This 
sequence of events happens even though there exists a fuel gauge monitoring the 
amount remaining in the cruiser. A possible explanation for the subject not monitoring 
the fuel level is that while the cruiser is in orbit, the subject is solely concentrating on 
the collection of comnaodities. Thus, the only gauges of concern are the cruiser's 
storage capacity gauges. In addition, since no thrusts arc being applied to Star Cruiser, 
no fuel is consumed and therefore there is no need to monitor the fuel gauge. If the 
cruiser is not in orbit, then the subjects appeared to be more concerned with steering the 
cruiser through the solar systems and galaxy in order to avoid firom crashing into any 
suns and/or to make sure it is heading back to Star Base. Very little attention was given 
to the fuel consumption gauge. Thus, if only a small amount of fuel was present, the 
subjects were likely not to notice, and after performing several thrusts the cruiser would 
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explode. This behavior occurred frequently whenever the subjects had difficulty 
controlling the speed of the cruiser. Since no cues exist which convey this speed to the 
user (refer to Place Star Cruiser Into Orbit) , it sometimes took several attempts to get 
the cruiser traveling at the proper docking speed. Docking the cruiser, of course, may 
only involve small thrusts, but they can be numerous. As a result again, the cruiser 
would run out of fuel and explode. 

This problem results from the perceptual cues' overspecification of action. Too 
man y cues exist independent of one another. The user is unable to efficiently monitor, 
identify, or notice all cues. Therefore, the information that is available to the user 
should somehow be restructured. In doing so though, no additional effort, mentally or 
physically, should have to be exerted by the user in order to successfully determine 
when to dock Star (Truiser at Star Base. One possible method for accomplishing this 
restructuring of information is to reduce the number of independent cues. For example, 
combining the information presented by multiple cues into one cue should help the user 
in deter minin g when to perform this action. Whatever the method, though, care must be 
taken that the modification does not result in the action becoming underspecified, 
mismatched, or even more overspecified than before. 

Release St ar Cruiser From Star Base 

The subjects considered this action to be the easiest which involves movement 
of Star Cruiser. Not only is it easy to perform the action, applying a thrust to a 
stationary object, but determining when to perform the action is simple as well. Unless 
the amount of time remaining in the mission is so low that nothing can be accomplished 
or all commodities within the galaxy have been collected, the subjects would remove 
Star Cmiser from Star Base immediately after it had docked. 
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Since the users cannot perform any other actions that affect the status of the 
galaxy while the cruiser is docked at Star Base, they understand that in order to 
continue achieving their goal of collecting commodities, they need to pull the cruiser 
away from the base and have it travel towards one of the solar systems. This constraint 
serves as a forcing function (Norman, 1988) which guarantees the users will perform 
the action if they are going to better their performance. In a sense then, the requirement 
to remove the cruiser from the base in order to affect the status of the galaxy is enough 
to inform the users that they should perform this action. Therefore, no additional 
perceptual cues are required. 

The only time this action would not be available to the user is if the current 
view is of a solar system. Only when viewing the gilaxy is the user capable of 
removing Star Cruiser from Star Base. In a view of a solar system, though, the cruiser 
would not even be displayed. Thus, the absence of a displayed Star Cruiser signifies to 
the user that a thrust cannot be applied to the cruiser to move it away from the dock. 
Whenever viewing the galaxy, however, the user may easily apply the thrust and pull 
Star Croiser away from the base. Since it was apparent that the subjects consistently 
knew about the availability of this action, no further perceptual cues are warranted. 

Change View To Galaxv/Solar System Map 

Whenever subjects performed this action, it was to gather information which 
would help determine Star Cruiser’s next movement. With the cruiser in the galaxy 
view, subjects would select a view of a solar system to assess the amount of 
commodities available on its planets. Different solar systems would be selected until 
the subjects determined that one contained enough commodities to justify sending the 
cruiser there. The cruiser’s next movement would then be towards that particular solar 
system. If the cruiser was present in the solar system, most always in orbit, the subjects 
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often selected the galaxy view in order to determine which side of the system the cruiser 
should exit from. The subjects who performed this action for this reason found it 
necessary to do so since the thrusts they applied to the cruiser to break it out of its orbit 
often sent it flying out of the solar system uncontrollably. Information obtained fix»m 
viewing the galaxy, such as the location of neighboring solar systems or the location of 
Star Base, often helped the subjects to either prevent the cruiser from sailing into 
another solar system and crashing into it's sun or find the shortest route for the cruiser to 
return to Star Base or to travel to another solar system. Often these alternate views are 
selected while the subject is waiting for the completion of another action (i.e., tools 
collecting commodities, cruiser traveling through galaxy, cruiser docked at Star Base). 

Though some subjects took advantage of this task in this fashion, some do not. 
Even though they were told about the option of different views and how to access them, 
some subjects would not use this feature. This lack of use may be because no 
perceptual cues are present to inform the user of the action's availability. There is 
nothing about the display which lets the user know that by selecting a sun in the galaxy 
map, or the sun in a local map, a different map can be observed. Though there are cues 
which indicate which solar systems may be viewed (the presence of the pie pieces on 
the suns), these still do not inform the user that the action can be performed. 

Additionally, cues which assist the user in determining when to perform this 
action are lacking. Only as a result of practice and habit docs the user perform this 
action. And then again, the multiple views are only employed if the user fails to 
remember the various states of the system. As a result, this action currentiy does little 
to help relieve the memory burden placed on the user. 

There are essentially no perceptual cues which specify this action. The 
subjects who did perform it though, generally had an easier time determining where to 
send Star Cruiser. Therefore, the action has merit and the display should support it. In 
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order to support the action, perceptual cues need to be introduced which will aid the 
user in determining if and when to perform this action, as well as whether or not the 
action can be performed. 


Summary 

The movement of Star Cruiser, as controlled by the users, is determined by a 
number of factors. These factors include the distance the cruiser has to travel, the 
configtu^tion of solar systems within the galaxy, the cruiser's amount of fuel, and the 
time remaining in the mission. All these factors play a vital role in the determination of 
where Star Cruiser needs to go and how it is going to get there. All of this information 
is readily available to the user via the display interface. In addition, this information 
serves as cues which contribute to the decision to perform other actions. Quite often, 
the user elects to perform an action but cannot, or will not, do so until another is 
performed. Many of the actions are related through such precedence constraints. For 
example, a subject may elect to remove Star Cruiser from orbit As a result, the user 
might first perform the action of viewing the galaxy to determine the cruiser’s 
destination and exit path and then actual remove the cruiser from orbit Such relations 
can be this simple, or they may be more complex. The action of returning commodities 
to Star Base may involve the user first deploying a probe, viewing the solar system 
containing that probe, steering the cruiser into that system, obtaining orbit deploying 
collection tools, retrieving the tools, removing the cruiser from its orbit (which itself 
may require multiple steps), steering it through the galaxy, and finally docking it at Star 
Base. 

The user must clearly understand these relations for acceptable performance to 
be obtained. The perceptual cues within the environment play a significant role in 
supporting this process. The cues should accomplish two goals. The first is that they 
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should always let the user know when a certain action may be performed. Secondly, if 
an action is unavailable, then cues should specify to the user what actions need to be 
performed in order to make the unavailable one possible. This process should repeat 
until the user is presented with an available action that will help achieve the availability 
of the original action. Whenever breakdowns in this process occur, then the existing 
perceptual cues should be investigated and the determination made about whether new 
cues should be used, current cues should be eliminated, or multiple cues should be 
combined. 

The preceding discussion concerning Star Cruiser’s possible actions 
attempted to accomplish this type of investigation. Experienced subjects were 
examined to determine when a particular action was likely to be performed. The 
subjects' abilities to perform these actions were then evaluated. Possible explanations 
for the ease or difficulty of selecting each action, based on the perceptual support 
available within the Star Cruiser environment, were also offered. As a result, areas of 
improvement can now be suggested. These improvements should concentrate on 
improving Star Cruiser's perceptual environment so that, where it failed before, the 
perceptual guidance available will specify what actions the users should do as well as 
whether those actions can be performed. 
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CHAPTER Y 

PERCEPTUAL ENHANCEMENT OF THE STAR CRUISER DISPLAY 


Upon review of the ecological task analysis for the Star Cruiser task, two 
pieces of information concerning perceptual cues become apparent for each task. The 
first concerns what cues are available to support a user's particular method for 
performing a task. The second concerns what cues exist to indicate to the user the 
availability of each task. With this knowledge, one may examine the existing display 
interface and propose certain changes to it which will help improve overall task 
performance. Proposed changes to the Star Cruiser's interface are presented below. 
Possible improvements concerning the nine originally analyzed tasks and other actions 
which may have surfaced during the task analysis are suggested. After each section is a 
brief description of the actual change, if any, made to the display interface. 

Deploy Collection Tools 
Proposed Di splay Enhancement 

An ideal display for Star Cruiser would be a display which indicates to the 
user which tools to deploy as well as when they should be deployed and where they 
should be sent This support, however, would most likely require aiding that is beyond 
the scope of perceptual enhancements (as defined in the context of Ecological Task 
Analysis). Some problems, however, may be eliminated. One problem was the 
confusion experienced by the users in trying to remember which tools collect what 
commodities and which types of planets they may be sent to. Simple modifications can 
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be made to the display interface to help alleviate this problem. This assistance may be 
accomplished by incorporating color coding into the part of the display where the 
various tools are located for selection. An example of such coding would include 
outlining the tools with the same color as the type of information that they collect. The 
background behind the tools can also be changed to match the color of the type of 
planets the tools can visit. These changes should help to reduce the confusion that 
users, especially novices, may experience. An additional enhancement that may be 
nnade to the display is to dim the tools available for selection whenever it is impossible 
for the user to do so. Though whether or not a tool could be deployed was very rarely 
an issue for the users, the possibility of confusion does exist and this change should 
help prevent such a problem from arising. Display enhancements concerning this task 
arc also discussed in E)etermining Star Cruiser Mode which occurs later in this chapter. 
Actual Displ ay Enhancements 

Several enhancements were made to the display interface to support this task. 
The first involved drawing some of the collection tool icons on a colored field (see 
Figure A-4). The science ships and miner ships, the two collection tools which can only 
be deployed to life-supporting planets, were placed on a green field, the same color as 
the life- supporting planets. This change was done to indicate to the user that the 
science ships and miner ships could only be deployed to the green planets. The other 
collection tools were left unchanged in this regard since there arc no restrictions on 
where they may be sent 

Background colors were added to the collection tools in order to indicate 
which type of commodity, data or resources, each tool gathered (see Figure A-4). The 
satellites and science ships each possess a light pink (to represent data) background 
while the robot miners and miner ships both have magenta (to represent resources) 
backgrounds. These colors are the same as those used to identify the presence of data 
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and resoiirces on the planets. This similarity should then aid the user in determining 
which collection tools should be deployed in order to gather a particular type of 
commodity present on the planets. 

A third enhancement involved crossing out the collection tools whenever they 
could not be deployed, such as when the cruiser was not in orbit. A yellow ‘X’ was 
placed over the tool icons when they could not be deployed (see Figure A-4). The tools 
were still visible, however, underneath the *X\ Since the original Star Cruiser display 
also used a yellow ‘X’ to show when a tool was out of commission (due to an 
overloaded Star Cruiser), the enhanced display incorporated a red *X’ to let the user 
know when this situation had happened and the services of a tool for the remainder of 
the session had been lost 

Another enhancement to the display was that the string drawn by the user 
during a tool deployment would change color when a possible target was acquired. 
When the user first selects a tool for deployment clicks the mouse on the cruiser, and 
then drags a deployment string out firom it the string is colored white. As soon as the 
mouse arrow is over a planet to which the tool may be deployed, however, the string 
changes color to green. This change in coltn* infOTtns the user to release the mouse 
button in order to deploy the tool. This display alteration was done in order to reduce 
the anoount of deployment errors that occur because the user was unable to click on a 
planet during this task. 


Retrieve Collection Tools 
Proposed Di splay Enhancements 

Three crucial problems with retrieving the collection tools were discovered by 
performing the task analysis. The first was the difficulty in selecdng a particular tool 
for retrieval whenever multiple tools were located at the same planet This problem was 
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due to the tools being overlapped near one comer of the planet. As a result, the tools 
could only be retrieved in a pre-specified order. This constraint was sometimes 
undesirable. Therefore, by depicting one tool in each comer, the user can select which 
tools to retrieve without having to retrieve any others beforehand. 

The second problem was the disappearance of the perceptual cues indicating 
the amount of data or resources contained on a planet once the cmiser collected that 
information. The user would thus have to remember how much each tool had collected 
so as not to overload the cmiser when the tools were retrieved. In order to reduce the 
amount of memory demands in such a task, other cues should replace those that vanish 
to indicate that the information has been collected and how much had been collected. 
Using "ghost images," replicates of representations which are a different color than the 
originals, in place of the pie pieces once they have disappeared would serve such a 
function. 

Lastly, confusion was often present due to the lack of a direct relationship 
between how much information was collected from planets (pie graphs) and how much 
Star Cruiser's load increased (bar gn^hs). Incorporating similar types of graphs to 
represent the two pieces of information would provide important perceptual cues to the 
user in determining whether or not to retrieve a particular tool. This goal may be 
accomplished by either changing the pie graphs on the planets to bars, or by changing 
the bars representing cruiser capacity to pie graphs. 

Actual Displ ay Enhancements 

One enhancement which supports deploying collection tools also supports their 
retrieval. As described before, the string which is dragged from the cmiser during 
deployment changes color when the target is hit This enhancement also aids in 
retrieving a tool. After a deployed tool has been selected, as soon as the white retrieval 
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string lands on the cruiser it will change to green. Thus, as with deployment errors, the 
number of retrieval errors should be reduced due to this change. 

In addition, ghost images were added to the planets in the local maps (see 
Figure A-3) and to the suns in the global maps (see Figure A-6) to indicate how much 
data or resources had just been collected. As data is collected, the light pink pie pieces 
on the planets or suns are replaced by white pie pieces. Black pie pieces appear in place 
of the magenta ones as resources arc collected. As a result, the user no longer needs to 
remember how much data and resources were present before they were gathered by the 
collection tools. 

The gauges representing Star Cruiser’s capacity were also altered (sec Figure 
A-4). Empty planets now represent how much Star Cruiser can carry. One planet holds 
the maximum amount of data and resources of an orbiting planet The capacity of Star 
Cruiser is therefore represented by as many planets as would be needed to fill up the 
cruiser. These planets are filled with pic pieces representing the total amount of 
commodities brought aboard Star Cruiser just as the planets orbiting the suns arc 
initially filled. This enhancement, along with the preceding one, was done to reduce the 
chance of the user overloading the cruiser by retrieving a tool which had collected to 
much da t^ or resources. For example, when the user retrieves a collection tool, it is 
known how much the tool has gathered by the appropriate ghost image of the pic pieces 
on the planet The user also is able to sec if that pie piece will "fit" into the cruiser by 
noticing if there is enough room in the planet gauges. If so, the user knows that the tool 
may be retrieved without overloading the cruiser. As a result, a more direct relationship 
exists between the pictorial representation of commodities on the planets and the 
amount on board Star Cruiser as depicted in the capacity gauges. 
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Deploy Probe 

Proposed Di splay Enhancements 

The deployment of a probe is yery similar to that of a collection tool. 
Essentially, the only difference is that a probe's deployment string is dragged to a sun 
whereas a collection tool's string is dragged to a planet As a result, seyeral of the 
possible enhancements which were directed at the deployment of a collection tool may 
also be applied to that of a probe. These include indicating when a probe cannot be 
deployed and the changing of the deployment string's color. Other possible 
enhancements made to the interface which inyolye this task arc discussed in 
Determining Star Cmiser Mode. 

Actual Displ ay Enhancements 

Some of the enhancements used to assist the user in deploying collection tools 
may also be used in deploying probes fev similar reasons. These include changing the 
deployment string color to green when the target, in this case a sun, has been hit and 
crossing out the probes when they cannot be deployed (Star Cruiser is docked at Star 
Base). 


Retrieye Probe 

Proposed Di splay Enhancements 

As discussed in the task analysis, the current display interface presents possible 
confusing perceptual cues with respect to retricying probes. The displaying of a probe 
next to a sun in the global map may lead a user to try to retrieyc it eyen though Star 
Cruiser is not in orbit in that particular solar system. In order to ayoid such confusion, 
the cue should be altered. One such remedy is to reduce the brighmess of the picture of 
the deployed probe in the galaxy yiew. This modification should still present the same 
information as before, that a probe has been deployed to a system, but it should also 
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reduce any tendencies that the users may have to try to retrieve it when the proper 
conditions have not been satisfied. 

Actual Displ ay Enhancements 

Just as deploying collection tools and probes shared similar display changes, so 
too do retrieving collection tools and probes. The retrieval string also changes to green 
to indicate to the user when to release the mouse button in order to retrieve a probe. 

Place Star Cruiser Into Orbit 
Proposed Di splay Enhancements 

Some of the problems involving placing Star Cruiser into orbit that became 
apparent from the task analysis were associated with trying to identify the location of 
the ninth orbit and when the cruiser was at the proper speed for obtaining orbit. Even 
though various perceptual cues already exist to aid the user in locating the ninth orbit, 
they are not direct and thus an additional cue should be added. One such possible cue 
involves highlighting the orbit in some fashion. Using a dotted line to trace the ninth 
orbit, for example, would allow the user to easily locate it A different form of 
highlighting can be employed to indicate when the cruiser is traveling at the proper 
speed for obtaining orbit When the speed is slow enough in the solar system view, the 
ship may either change color or, similar to when the ship is in orbit, become outlined 
using some color. Not only will such an addition of perceptual cues allow the user to 
identify when the cruiser is likely to obtain orbit but it will also allow the user to notice 
when Star Cruiser may be captured by the orbit even though it is undesirable. The 
cruiser’s speed or direction may then be changed to ensure that it does not enter the 
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Actual Displ ay Enhancements 

In order to assist the use in determining when the cruiser was crossing over the 
ninth orbit, a white 'X' indicating the orbit’s position was added to the display (see 
Figure A-5). This 'X' follows the path of the ninth orbit, always placing itself in a direct 
line with the sun and the cruiser. This way, as the cruiser moves closer to the orbit, it is 
also getting closer to the marker. Therefore, when Star Cruiser is crossing the ’X,’ it is 
also intersecting the orbit Therefore, if the cruiser is going at the proper speed. Star 
Cruiser will go into orbit as soon as it passes over the 'X.' 

A second enhancement was also added to the display to assist the user in this 
task. Star Cruiser, when it is traveling at the proper speed for obtaining orbit will be 
outlined in purple, much like it is outlined in green when it is in orbit From this change 
in color, the user will know if the cruiser will enter orbit as it crosses the ninth orbit or 
if it will not because it is traveling too fast 

Remove St ar Cruiser From Orbit 
Proposed Display Enhancements 

One problem that often occurred when users attempted to remove Star Cruiser 
from orbit was that it was done so accidentally. This occurrence resulted because they 
did not realize that a thrust was being applied to the ship. This problem, along with 
possible solutions, are further discussed in Determinin g Star Cruiser Mode. Users also 
experienced difficulty with this task whenever they applied too big of a thrust to the 
cruiser. As a result, the cruiser would leave orbit at an uncontrollable speed, leave the 
solar system, and sometimes enter other systems. This sequence of events increased the 
chance of Star Cruiser colliding with a sun and exploding. Even though this may be the 
case, it is not apparent whether or not adding any additional perceptual cues to reduce 
this risk would have any benefit on performance since this sequence of events only 
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occuiTcd infrequently. Such cues, though, may include changing the color of the cruiser 
or causing the cruiser to blink on and off whenever a thrust string applied to it is large 
enough to remove it from orbit. Also, an auditory tone may be used to indicate if the 
cruiser’s direction of travel will send it into a neighboring solar system. 

Actual Displ ay Enhancements 

With the exception of those enhancements discussed in Determining Star 
rniiser Mode, none were added to the display interface to support this task since the 
ecological task analysis failed to indicate their possible benefit on user performance. 

Dock Star Cruiser At Star Base 
Proposed Di splay Enhancements 

When to perform this action is sometimes not clear. Of the three factors which 
may cause a user to return the cruiser to Star Base (loaded cruiser, no fuel, session time 
expiring), the only one which seemed to go unnoticed repeatedly was the amount of fuel 
Star Cruiser had remaining. In order to make the fuel gauge more noticeable, several 
changes to this perceptual cue should be made. One is to change it's color. Currently, it 
is dif ficult to discriminate between the fuel gauge and the display's background due to 
their colors. As a result, the user must look carefully at the gauge to determine the fuel 
leveL Chan ging the color, and even using multiple colors to represent different fuel 
levels, should help to make the fuel gauge more discriminable from the rest of the 
display. In addition, the information provided by the gauge should be more 
interpretable (i.e., given how much fuel is remaining implies how big of a thrust can be 
applied). One other enhancement to the display concerning the docking of the cruiser at 
Star Base is to change its color when it is traveling at the proper docking speed. This 
display change is essentially the same as the enhancement suggested for obtaining orbit 


with Star Quiser. 
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Actual Displ ay Enhancements 

Star Cruiser becomes outlined in purple so that the user can determine when 
the cruiser is traveling at the proper docking speed. This display enhancement is similar 
to the one described in Place Star Cruiser Into Orbit Since the cruiser can never dock 
at Star Base in the local map, and since it can never go into orbit in the global map, 
there should be very little confusion in determining the meaning of this cue at a given 
dme. 


Release Star Cruiser From Star Base 

No significant problems resulting from a lack or misuse of perceptual cues 
concerning this task were noticed during the task analysis pcrfonned on Star Cruiser. 
Therefore, there is no need to make any changes to the display interface in order to 
better support the releasing of the cruiser from Star Base. 

Change Vie w To Galaxv / Solar System 
Perceptual cues which informed the user of the availability of this task or 
assisted the user in determining when to perform this task were discovered to be 
lacking. This task could be cased by presenting a snudler view of the global map to the 
user whenever the cruiser is in a solar system. Similar "radar views" of solar systems 
could also be used when Star Cruiser is traveling through the galaxy. These changes, 
however, are not in accordance with the perceptual enhancement approach used in this 
study. Therefore, without the addition of any perceptual cues, this task should be 
thoroughly discussed with users during training sessions in order to familiarize them 
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Choose Star Cruiser's Destinarion 
Proposed Dis play Enhancements 

This task involves the user determining to which solar system to send Star 
Cruiser. It does not include determining when to send Star Cruiser to Star Base as that 
is covered by another task. Since the user's primary goal is to collect as many 
commodities as possible from the solar systems and return them to Star Base, any 
display enhancements aimed at assisting the user with this task may result in a 
significant increase in performance. In attempting to aid the user in making this 
determination, the display offers as a perceptual cue the color of the suns. As the 
scenarios are currently programmed, the color of the suns indicate the amount of 
conuiKxlities contained in the solar system given the possible amount that may be 
contained there. This calculation is based on the number of planets present within the 
solar system. As a result, one solar system may actually possess more data and 
resources than another even though the color of its sun indicates otherwise. An 
improvement would be to program the scenarios so that the color of the sun indicates 
the amount of commodities available in the solar system relative to the amount 
available in the galaxy. This simple change should facilitate the user's determination of 
which solar systems contain the most data and resources and are worthy of attention. 
Actual Display Enhancements 

Two changes were made to the display interface to help better support the user 
in determining to which solar system to send the cruiser. The first involved a change in 
an existing perceptual cue. As mentioned previously, the suns' colors could be 
programmed to better indicate the systems which contained the most commodities. 

This use of color was done by assigning a brighter colored sun to those solar systems 
which contained a higher percentage of the total amount of data and resources in the 
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entire galaxy. Those systems which did not contain many commodities were therefore 
given darker colored suns. 

Another change to the interface resulted in the addition of a new cue. A blue 
X’ was placed over the sun of any solar system in the global map to indicate that it was 
completely empty (see Figure A-6). A solar system was considered empty if it did not 
contain any commodities, collection tools, or probes. Planets were not considered. 

This enhancement would inform the user of which solar systems could be avoided. The 
'X* was blue to avoid any confusion that may arise due to other colors used in the 
display, especially the colors of the suns. 

Determin ing Star Cruiser Mode 
Proposed Di splay Enhancements 

Users often had difficulty determining when an action performed on Star 
Cruiser would result in a thrust being applied or whether the action would result in the 
deployment of a tool. Such problems arose, for example, when subjects were 
unsuccessful at selecting the cruiser once a collection tool had been chosen for 
deployment. Other times this problem was due to a probe being selected and the user 
not realizing it Except for the highlighted tool at the top of the screen, no cues ever 
existed to indicate to the user whether a string pull from the cruiser was for a collection 
tool, probe deployment, or for a thrust. Incorporating auditory tones whenever the user 
performs an unsuccessful action (i.e., missing selecting the cruiser after choosing a tool 
to deploy) and/or altering the color of the cruiser to indicate the type of action mode the 
cruiser is in (i.e., a string pull will deploy a tool) should provide the user with beneficial 
perceptual cues that will help to eliminate these problems. 
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Actual Disp lay Enhancements 

Star Cruiser was outlined in red whenever it was in a deployment mode. For 
example, if the user has selected a collection tool or a probe for deployment, the cruiser 
would become outlined. If for some reason the tool or probe is no longer selected, the 
red outline disappears. This enhancement to the display interface should assist the user 
in determining whether performing a string pull on the cruiser will deploy a tool or 
probe or if it will apply a thrust to it In other words, will the cruiser be in a deployment 
mode or a thrust mode. 


Summary 

In summary, the actual enhancements made to Star Cruiser's display interface 

were: 

1. Place collection tools on a colored field to indicate any planet restrictions. 

2. Add background colors to tools to indicate type of commodity collected. 

3. ‘X-out’ tools when they cannot be deployed. 

4. Change color of deployment / retrieval string when it ends on a possible 
target 

5. Use ghost images as a reminder of what comnnodities have been collected. 

6. Change Star Cruiser capacity gauges so that they are similar to pie pieces 
on planets. 

7. Add white ‘X’ to help locate ninth orbit 

8. Add purple outline to Star Cruiser to indicate speed. 

9. Change definition of colors of suns. 

10. Place blue ‘X’ over suns of empty solar systems in global map. 

1 1. Add red outline to Star Cruiser to indicate action nnode. 
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Table 5-1 lists the proposed display enhancements according to the task they are 
designed to aid. One should note that these arc not the only enhancements that could 
have been made. Many other possibilities existed as detailed above. In deciding which 
changes to incorporate, two factors were considered. The first was the difficulty 
associated with making changes to the Star Cruiser software. The other factor was the 
goal to empirically evaluate the ecological task analysis framework. Only changes that 
were suggested by the ecological task analysis process were incorporated. 



Table 5-1. Proposed Display Enhancements 


Deploy Collection Tool 

• Place Tools On A Colored Field 

• AdW Background Colors To Tools 

• 'X’-Out Tools Which Cannot Be Deployed 

• Change Color Of Deployment String 

Retrieve Co llection Tools 

• Change Color Of Retrieval String 

• Use "Ghost Images" For Collected Commodities 

• Alter Star Cruiser Capacity Gauges 

Deploy Probe 

• 'X’-Out Tools Which Cannot Be Deployed 

• Change Color Of Deployment String 

Retrieve Probe 

• Change Color Of Retrieval String 

Place Star Cruiser Into Orbit 

• Add White 'X' To Locate Ninth Orbit 

• Add Purple Outline To Indicate Cruiser Speed 

Remove Sta r Cruiser From Orbit 

• None 

Dock Star Cruiser At Star 

• Add Purple Outline To Indicate Cruiser Speed 

Release Star Cruiser From Star Base 

• None 

Change Vie w To Galaxv / Solar System 

• None 

Choose Star Cruiser's Destination 

• Change Definition Of Colors Of Suns 

• Use blue 'X* To Indicate Empty Solar System 

Determinin g Star Cruiser Mode 

• Add Red Outline To Indicate Mode 



EMPIRICAL EVALUATION OF PERCEPTUAL ENHANCEMENTS 


The Ecological Task Analysis frameworic has proven capable of suggesting 
possible perceptual enhancements to the display interface of Star Cruiser. This fact 
does not, however, imply that these suggestions will improve performance on the task. 
In order to determine how effective the proposed enhancements are at improving task 
performance, a new version of Star Cruiser was created which incorporated the 
suggested augmentations. As a result, it was possible to conduct an experiment which 
compared users’ performance on the original Star Cruiser to that of users on the 
enhanced version. 


Experimental Method 

Star Cruiser Simulations 

Two versions of the Star Cruiser simulation were used to test the 
effectiveness of the display enhancements suggested by the ecological task analysis. 

The first. Star Cruiser vl.ll r4 , was the original task used to perform the initial 
exploratory analysis. The display enhancements were made to a copy of this version to 
create a second, enhanced version used during this experiment With the exception of 
the enhancements, these two versions were alike in every way. 

Both versions operated on Macintosh Ilci computers, each with a standard 
Macintosh mouse. Two of these computers were used so that multiple subjects could be 
tested at the same time. The displays for both were 19” color monitors. The only 



68 


difference between the two sets of hardware was that one operated using version 6.0.5 
of its system software while the other ran with version 7.0. This difference did not, 
however, affect the operation of the simulation in any significant way. 

Subjects 

A total of sixteen subjects were used during this experiment. This pool 
consisted of ten males and six females. Two groups, one for each version of Star 
Cruiser, were created consisting of eight subjects each, five male and three female. All 
subjects were undergraduate students at the Georgia Institute of Technology. They 
were also all right-handed. The subjects were informed that for their full participation 
in the experiment, they would be compensated $25, $5 an hour for five hours over a two 
week period. They were also told that there were two $25 bonuses, one for each group, 
to the individuals who had the highest average score after the two weeks (not including 
scores from training). 

Sessions (Scenarios! 

The experiment occurred over a period of two weeks with the subjects 
performing the task for a total of ten days (Monday through Friday of both weeks). The 
first two days were used to familiarize the subjects with Star Cruiser and allow them to 
practice before the actual data gathering began. During these first two days, experts at 
performing the task were present to answer any questions and to coach the subjects 
whenever they were doing something fundamentally wrong. Only if they 
misunderstood some basic part of the task were they given advice. No tricks or 
important insights were revealed to theiiL The training periods lasted for one half hour 
each day. After the second day, the subjects were given two sessions a day for a total of 
eight days. No assistance of any kind was given to them at this point. Each session 
lasted a maximum of ten minutes. 
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There were four different types of sessions that the subjects experienced. No 
matter the type though, the subjects were instructed to try to achieve as high of a score 
as possible. The first type was the training sessions. These were then followed by a 
total of twelve (over six days) normal sessions which had them simply performing the 
task as best they could. 

On the ninth day, the subjects were asked to perform an additional task. While 
performing the Star Cruiser simulation, for both sessions that day, the subjects were 
required to listen to a tape of simple math problems (i.e., 14+7, 24-10, 8*4, 42/7). It 
was their goal to answer, verbally, as many correct as possible. In order to ensure that 
they performed as best they could on both tasks, the subjects were informed that their 
score on Star Cruiser for the two sessions would be multiplied by their percentage of 
correctly answered math problems when determining who received the bonus money. 
The motivation for these sessions was to test one of the main goals of ecological task 
analysis. That goal is to reduce the cognitive demands placed on the user. By 
monitoring the performance of the subject groups at the Star Cmiser task while 
attempting the concurrent task of answering math problems, the effectiveness of the 
display enhancements, and thus the task analysis, can be determined. 

On the last day of the experiment, the subjects were required only to perform 
the Star Cruiser task, it was, however, presented to them using whichever display they 
were not familiar with. In other words, those who had (niginally been using the 
enhanced display version were given the original and vice versa. The goal of these last 
two sessions was to determine how performance would be affected if the additional 
cues provided in the enhanced display were renx>ved from the enhanced display or 
added to the original display. 

Eighteen different scenarios were created for this experiment, one for each day 
of training and two for every day thereafter. The configuration of the galaxy was 
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different for each scenario. Factors that were varied included the number of solar 
systems in the galaxy and their locations, the size of the sun in each solar system, the 
number and type of planets in each solar system as well as their orbits, the amount of 
data and resources on each planet, the location of Star Base, and the starting location of 
Star Cruiser. The configuration of Star Cruiser, however, remained consistent 
throughout all scenarios. These constants included such factors as the number of probes 
available, the number of each type of collection tool available, the fuel consumption 
rate, the data and resource storage capacity of the cruiser, and any factor within the 
scenario file which affected Star Cruiser’s movements. In addition, the points awarded 
for data and resources returned to Star Base were equal and consistent throughout all 
scenarios. The scenarios were developed to be of approximately equal difficulty. With 
the exception of any differences that may arise due to the display enhancements, each 
subject was presented the exact same scenario during the same session number. 

Hypotheses 

Overall Performance 

The purpose behind this experiment was to determine if the Ecological Task 
Analysis fr ame work could be used to suggest possible display enhancements for a task 
that would improve a user’s performance. For the Star Cruiser task, there are several 
different measures of performance that one could investigate to determine if the task 
analysis was useful in this regard. The simplest measure is the user’s score on the task. 
High scores are typical of those users who are able to perform the task well. It is 
believed then, that if the suggested display enhancements did contribute to better user 
performance, the scores will reflect it. Therefore, higher scores would be expected from 
the group of subjects using the enhanced version of Star Cruiser as compared to those 
using the original. 
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It is also thought that the subjects on the enhanced version would reach their 
maximum performance level faster. In other words, they will learn the task faster than 
the group using the original version. This belief is because one of the goals of the 
ecological task analysis is to bring to the display surface more useful information about 
the task and the task’s ever-changing system states that the user can utilize while 
performing the task. Since the “enhanced” user now has additional sources of 
information that the “original” user does not, the enhanced user should be able to learn 
more quickly how to perform the task as well as any tricks that may help to improve 
performance. It is therefore expected that the enhanced version would result in users 
reaching higher scores faster than the original version. Only sessions 3 through 14 were 
used in determining the display enhancements effects on learning and overall 
performance since sessions 1 and 2 involved training and sessions 15 through 18 
involved slightly different tasks for the user. 

Individual P erformance Measures 

The enhanced version of Star Cruiser is made up of many individual display 
changes. How each change affects user performance can be measured to some degree. 
By gathering such data, typically in the form of performance errors or time to perform 
an action, one may see exactly which enhancements had the biggest influence, if any, 
on user performance. Such results may reveal areas of the simulation where further 
display enhancements, or even more significant changes to the simulation as a whole, 
may be required. These results also may indicate those areas where efforts should no 
longer be placed since they appeared to have no, or negative, effects on user 
performance. Following is a listing of each display enhancement, how its effect on user 
performance was measured, and how it was expected to affect performance. It should 
be noted that for some of the measurements, there are limits to how well they reflect 
user performance. 
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Colored Field / Background Colors Of Collection Tools. The first 
performance measure actually encompasses two of the display changes. The first 
change was placing the science ships and miner ships on green fields to indicate that 
they may only be deployed to life-supporting planets. The other enhancement was 
providing background colors to the collection tools to indicate whether they gathered 
data or resources. Both of these changes were implemented to aid the user in 
determining the functions and limitations of the tools. One way to measure how 
effective these enhancements were is to check how often a user made an error by 
selecting the wrong tool. The only true check of this type of error is to keep track of 
how many times a tool is selected immediately after one has been chosen, even if it is 
the same tool. This procedure may, however, not account for only selection errors since 
other events may dictate choosing a different tool or warrant selecting the same tool 
over again. In addition, other occurrences of the user selecting the wrong tool may not 
be counted. The user of the enhanced interface should, though, have fewer selection 
errors than the original user because of these two display changes. 

- Out Tools When They Ca nnot Be Deployed. This display enhancement 
was intended to reduce occurrences of users trying to select and deploy a tool when it 
was not possible to do so. To measure this error, the number of times a collection tool 
was selected when the cruiser was not in orbit or the user was viewing the global map 
was counted. Each time the user attempted to deploy a probe when Star Cruiser was 
docked was also added to this value. This performance measure is relatively 
straightforward and the user working with the enhanced display should have fewer of 
these errors. 

Change Color Of Deployment / Retrieval String . While performing the task 
analysis on Star Cruiser, it was observed that users had difficulty determining when 
they were able to release the mouse button over a target during a tool deployment or 
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retrieval. This display enhancement was intended to prevent this problem. The 
simulation was able to count the amount of deployment / retrieval errors that occurred 
because the user missed the planet/sun/cruiscr while performing the action. Since the 
enhanced subjects were being informed of when they had acquired the target, they 
should have had fewer deployment / retrieval errors than those subjects using the 
original display. 

Use Ghost Images . This display enhancement was incorporated into the new 
version of Star Cruiser in order to reduce the chance of the user overloading Star 
Cruiser with too much data or resources. If this situation ever did occur, then the tool 
which the user attempted to retrieve would be labeled out of commission in the 
selection bar at the top of the display (red ‘X’ in enhanced display, yellow ‘X’ in 
original). Therefore, simply counting the number of tools that were out of commission 
during a session would provide a sufficient measure of the effectiveness of these 
enhancements. Since subjects were provided with additional and clearer information 
regarding the collection tools and Star Cruiser’s current load of data and resources, 
those subjects who were given the enhanced display version would be expected to have 
fewer tools out of commission than those working with the original display. 

Alter Star Cruiser Capacity Gauges . Remembering that the goal of the 
subjects is to score as many points as possible, it seems reasonable to assume that, on 
the average, subjects would attempt to return as much data and resources as p>ossible 
each tiine they return to Star Base (this assumption was later determined to be 
reasonable by observing skilled subject behavior while performing the task). Since 
those subjects working with the enhanced displays should be better informed as to how 
much more data and resources will fit on Star Cruiser and what collection tools will 
accommodate that amount, they should be able to fill the cruiser to near capacity with 
commodities. Thus, they should be better able to unload, on average, more 
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commodities at Star Base during each docking. Therefore, keeping track of the average 
percent utilization of Star Cruiser’s total commodities capacity as it docks at Star Base 
provides a second performance measure for the effectiveness of these enhancements. 

White ’X’ For Ninth Orbit / Purple Highlight For Cruiser Sneed. The user’s 
ability to place Star Cruiser into orbit is another important performance factor. These 
two display enhancements should be included together when discussing this 
performance measure since the location of the ninth orbit and speed of the cruiser are 
both important factors to be considered while attempting to place the cruiser into orbit. 

It would be expected that those subjects who had help in locating the ninth orbit would 
find the task of placing Star Cruiser into orbit an easier one than those without any 
assistance. The group which used the enhanced display should then have required less 
time to place the cruiser into orbit compared to the other group. In addition, they 
should not have been required to apply as many thrusts to Star Cruiser in order to enter 
orbit The time and number of thrusts required to obtain orbit were therefore recorded 
each time the user had Star Cruiser enter a solar system. If the cruiser exited the solar 
system before it obtained orbit, the counts were abandoned. This method was by far the 
best way to measure this factor with one exception. The cruiser could enter a solar 
system and travel to near the ninth cMijit before the user ever acted upon it. Since data 
was not recorded by the simulation unless a mouse click occurred, the exact time the 
cruiser was in the solar system may not be accurate. This factor was, however, 
consistent for subjects in both display groups. 

Purple Highlight For Cruiser Sneed (Docking). As discussed previously, this 
display enhancement should allow the user to better monitor the cruiser’s speed, 
especially when attempting to dock. Therefore, since they had information concerning 
the cruiser’s speed, those subjects who used the enhanced display version should have 
required fewer thrusts and less time to dock the cruiser than those using the original 
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display. A good performance measure to determine the effectiveness of this 
enhancement was to keep track of both the number of thrusts and time required for the 
user to dock Star Cruiser. The counts began when it became apparent that the user was 
attempting to dock the cruiser. It was determined that if the cruiser passed within fifty 
pixels of Star Base, then it was probably the user’s intent to dock it. If the cruiser then 
entered a solar system or if the user performed some other action other than a thrust 
before the cruiser docked, the counts were reset Otherwise, the time and number of 
thrusts were continually tabulated until the cruiser docked. As with measuring the 
performance of a user attempting to place the cruiser into orbit the fact that data isn’t 
recorded until a mouse click occurs presents a slight problem. As a result, the user may 
have the intent to dock the cruiser and send it towards Star Base while it is outside of 
the fifty pixel boundary. Then, without ever having to apply a thrust beforehand, as 
soon as the cruiser passes over the base, the user may simply apply a quick reverse 
thrust to dock it This event would then be rectmled as a docking where there were zero 
thrusts and it took zero seconds. This drawback was present across all subjects and 
even though it may have occurred (it only rarely did so), this means of measuring the 
effectiveness of this enhancement was the best possible. 

rhange Meaning Of Tnlor Of The Suns. The enhanced display included this 
change in order to inform the user of those solar systems which contained the most data 
and resources. Therefore, subjects working with the enhanced version of Star Cruiser 
should have visited more, compared to those using the original version, solar systems 
that, on average, contained the most commodities. This amount includes the solar 
systems to which only probes were sent since what is being measured by this factor is 
the user’s performance at determining which systems contain the most data and 
resources and, in addition to actually having the cruiser visit a solar system, sending a 
probe to a system is a method for making such a determination. The actual performance 
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measure here is the average amount of data plus resources for all visited or probed solar 
systems. 

Blue *X’ For Empty Solar Systems . Due to this change in the display, subjects 
using the enhanced display version should have Star Cruiser visit empty solar systems 
less often than those using the original display. Counting the number of times the 
cruiser entered an empty solar system provides a measure of the effectiveness of this 
perceptual enhancement 

Red Highlight For Cruiser Mode. This display enhancement’s purpose was to 
inform the user about which nrode, thrusting or deployment. Star Cruiser was currently 
in. In other words, whether a string pull on the cruiser would result in its thrusting or in 
a tool being deployed. The enhanced display group, compared to the group with the 
original display, should make fewer errors in determining which nnode the cruiser is in. 
This performance measure was obtained by first noting when the user had selected a 
probe or collection tool for deployment It was then checked to sec if the user was 
unable to select the cruiser at the start of the deployment process. If so, if the user 
attempted to deploy the tool once more before selecting it again, then this happening 
would be counted as an error in knowing die cruiser’s mode (since missing the cruiser 
the first rime would switch it from the deployment mode to the thrust mode). The 
reported total occurrences of this error was the performance measure for this 
enhancement 
Math Sessions 

Sessions 15 and 16 required the subjects to answer the math problems as 
described earlier. If the display enhancements resulting from the ecological task 
analysis arc providing the user with new or additional information needed to perform 
the or bringing information at the depth level to the surface, then there should not 
be as great of a demand for cognitive resources as when the user was using the original 
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display. As a result, the subjects using the enhanced display version should be better 
able to handle the additional task of performing math problems while simultaneously 
performing the Star Cruiser task. This hypothesis will be supported if both groups of 
subjects continued to score as they had been before receiving the math problems, but 
the enhanced group was able to get a significantly higher percentage of the math 
problems correct than the original group. The hypothesis will also be supported if both 
groups score the same on the math problems but the original group’s Star Cruiser 
scores suffered to a significantly greater extent than the enhanced group’s. In addition, 
subject performance on each individual performance measure should be less affected by 
the additional task for those using the enhanced display compared to the other subject 
group. The Star Cruiser scores and other performance measures obtained during the 
math sessions will be compared only to those obtained during the inunediately previous 
two sessions, Sessions 13 and 14, since these are closest to die experience level that the 
subjects were at when they received the math problem sessions. 

Transfer Sessions 

Sessions 17 and 18, the last two, were used to determine if any performance 
transfer would occur between the two types of displays. Those subjects who had 
originally been using the unmodified version were presented with the enhanced version 
and vice versa. It was expected that the scores of the group which initially used the 
enhanced version, as well as performance as measured by the individual factors, would 
suffer since they were no longer presented the information conveyed through the 
improved perceptual cues, information that they had come to rely upon. The overall 
performance as measured by scores and the performance of the individual tasks of the 
other group, it was thought, would either remain constant or improve. Performance 
remaining constant would suggest that even though new information was being made 
available, the subjects either did not have enough practice at utilizing it or they were so 
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set in their methods of performing the task, that they ignored it. If their performance 
improved, it was believed to be due to the information provided through the improved 
perceptual cues. As with the math sessions, subjects’ performance on the transfer 
sessions was compared only to those for Sessions 13 and 14. 



79 


CHAPIEKYn 

RESULTS OF DISPLAY ENHANCEMENT EVALUATION EXPERIMENT 

As each subject performed the task. Star Cruiser recorded every action 
performed. This recording was accomplished by having the program write to a file the 
state of the simulation every time the mouse button was depressed or released. Each 
data entry included the time of the action, or event, recorded in sixtieths of a second. 
Also recorded were the type of event, the X-Y coordinates of the mouse pointer, and 
information concerning the status of the Star Cruiser including its X-Y coordinates, 
what solar system, if any, it was in, how much data and resources it had on board and 
how much it had unloaded at Star Base, and how much fuel was remaining. The entries 
also included information about which map was being viewed, and if applicable, of 
which solar system, and the status of each individual tool including where it was located 
if it was not on board the cruiser and, if it was in orbit, what orbit number it occupied. 

Each data file, as it was written, was compressed and therefore needed to be 
converted into a readable file using a separate program. Once deciphered, the data was 
read by another program which converted it into the form of the previously discussed 
perftnmance measures. This conversion was done for all but the training sessions. The 
results were organized into three groups depending on the type of sessions involved. 

The first series consisted of sessions 3 through 14, where the user simply performed the 
task, always using the same display version. Sessions 15 and 16, those which had the 
users answering math problems as well as performing the task, comprised the second 
group. Sessions 17 and 18 made up the final group. These sessions were where the 
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user switched display versions to perform the task. The results in each group were 
analyzed to determine if the mean results for the subjects’ overall performance and the 
set of ten detailed performance measures were statistically different between the two 
subject groups. An analysis of variance was performed on each set of results within 
each series of sessions to determine which variables had a significant effect (alpha = 
0.05) on user performance. The actual analysis was performed using the statistical 
software package SAS. The main effects tested were Session number. Display type, 
and Subject. Also of interest was the interaction effect between Session and Display on 
subject performance. Following are the results obtained from the analyses of variance 
for each series of sessions. The results for the individual performance measures have 
been ordered according to the display enhancement whose effectiveness they are 
evaluating. For a recap of what enhancements are evaluated by which measures, refer 
to Table 7-1. 


Sessions 3 - 14: Normal 

Overall Performance 

The mean score for subjects using the enhanced display version was 1 1350.6 
with a standard deviation of 5646.9. For those using the original display, the mean 
score was 9165.3 with a standard deviation of 5059.8. The trends of the scores for each 
subject group by session is shown in Figure 7-1. An analysis of variance revealed 
significant effects to be Session [F = 3.55; p = 0.(XX)2], Display [F = 10.(X); p = O.CX319], 
and Subject [F = 2.67; p = 0.0013]. 

Colored Field / Backgr ound Colors Of Collection Tools 

These enhancements’ effectiveness were determined by the number of times 
the subjects selected the wrong type of tool. Only one subject ever did so. That subject 
was using the original display version. Therefore, while there was no mean for the 


Table 7-1. Display Enhancements And Corresponding Performance Measures 


Display Enhancement 

Overall Performance 

Colored Field / Background Colors Of 
Collection Tools 

‘X’ - Out Tools When They Cannot Be 
Deployed 

Change Color Of Deployment / Retrieval 
String 

Use Ghost Images 

Alter Star Cruiser Capacity Gauges 

White *X’F(V Ninth Orbit 

Purple Highlight For Cruiser Speed 
(Orbiting) 

Purple Highlight For Cruiser Speed 
(Docking - Time) 

Purple Highlight For Cruiser Speed 
(Docking - Thrusts) 

Change Meaning Of The Color Of The Suns 
Blue *X’ For Empty Solar Systems 
Red Highlight For Cruiser Mode 


Performance Measure 
Mean Overall Score 
Select Wrong Tools 

Select Action Errois 

Deployment / Retrieval Errors 

Tools Out Of Commission 
Utilization Of Star Cruiser Capacity 
Time To Orbit (sec.) 

Thrusts To Orbit 

Time To Dock (sec.) 

Thrusts To Dock 

Commodities In Visited S<^ Systems 

Empty Solar Systems Visited 

Etrots In Determining Star Cruiser Mode 



Mean Display Group Overall Scores • Normal Sessions 
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Figure 7-1. Mean Display Group Overall Scores - Nonnal Sessions 
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enhanced version group, that for the original version was 0.021 with a standard 
deviation of 0.144. As a result, only the Subject effect [F = 0.0079; p = 0.0079] was 
found to be significant after performing an analysis of variance on the data. 

- Out Tools When They Cannot Be Deployed 

Rarely did a subject attempt to selea a collection tool or probe when it was not 
possible to do so. The mean number of attempts at doing so for those using the 
enhanced display was 0.271 with a standard deviation of 1.638 while for the original 
group it was 0.094 with a standard deviation of 0.358. The analysis of variance failed 
to find any significant effects. 

Change Color Of Deployment / Retrieval String 

This enhancement was incorporated to reduce the amount of 
deployment/retrieval errors that occur. As Figure 7-2 shows, there appears to be 
significant differences between the number of such errors committed by the original 
display subjects and those using the enhanced display. The mean number of errors for 
the original group was 13.67 with a standard deviation of 12.95 while the mean for the 
other group was 5.63 with a standard deviation of 5.45. The analysis of variance 
reveals that all three main effects tested. Session [F = 2.43 ; p = 0.0080], Display [F = 

1 10.22; p = 0.0001], and Subject [F = 31.15; p = 0.0001], were significant 
Use Ghost Images 

This display change was done to reduce the number of tools that go out of 
commission during a session. The analysis of variance showed that only the Session 
effect [F = 2.88; p = 0.0018] was significant for this performance measure. The mean 
number of tools that went out of commission for the enhanced display user was 0.396 
with a standard deviation of 1.100. The mean for the original display user was 0.292 
with a standard deviation of 0.7 10. Figure 7-3 graphs the mean number of tools that 
went out of commission for each group of subjects during sessions 3 through 14. 


Mean Number Of Dqjloyment / Retrieval Errors - Normal Sessions 
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Figure 7-2. Mean Number Of Deployment / Retrieval Errors - Normal Sessions 
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Figure 7-3. Mean Number Of Tools Out Of Commission - Normal Sessions 
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Alter Star Cr uiser Capacity Gauges 

In order to measure the effectiveness of this enhancement, the percent 
utilization of Star Cruiser’s capacity when docking was determined. The mean percent 
utilization exhibited by the enhanced display subjects was 0.663 with a standard 
deviation of 0.150. For those subjects using the original display, the mean percentage 
was 0.608 with a standard deviation of 0. 199. The trend over sessions of mean 
utilization for each subject group is shown in Figure 7-4. All effects tested during the 
analysis of variance proved to be significant These effects include the main effects of 
Session [F = 2.71; p = 0.0035], Display [F = 7.74; p = 0.0062], and Subject [F = 8.07; p 
- 0.0001], as well as the interaction effect of Session and Display [F = 2.44; p = 

0.0083]. 

White *X’ Fo r Ninth Orbit 

The primary purpose behind this enhancement was to reduce the amount of 
time that it took for the subject to place Star Cruiser into orbit From the graph in 
Figure 7-5, it appears to have done just that Specifically, the mean time to place the 
cruiser into orbit was 8.89 seconds with a standard deviation of 2.99 for those subjects 
who had use of the white ‘X’ enhancement Those who did not, the subjects using the 
original display, however, had a mean time to orbit of 15.79 seconds with a standard 
deviation of 10.25. The analysis of variance performed on this measure revealed that all 
three of the main effects. Session [F = 2.02; p = 0.0302], Display [F = 70.93; p = 
0.0001], and Subject [F = 10.28; p = 0.0001] were significant This finding is not 
sufficient, however, for concluding that this enhancement made it easier for the user to 
place the cruiser into orbit This display change assisted the subject with only one 
element required for completing this task: finding the ninth orbit. Cruiser speed is also 
an im portant element. Therefore, before one can state with any confidence that this 



Mean Uiilizaiion Of Star Cruiser Capacity - Normal Sessions 


87 



j(ii3edB3 josmQ jms JO uoiniznPn 


Figure 7-4. Mean Utilization Of Star Cruiser Capacity - Normal Sessions 
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Figure 7-5. Mean Time To Orbit (sec.) - Nonnal Sessions 
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change is responsible for the improvement in the time to obtain orbit, any enhancements 
involving the speed of the cruiser must be considered. 

Purple High light For Cruiser Speed (Orbiting) 

As just mentioned, this enhancement is critical in determining whether or not 
the enhanced display, when compared to the anginal, made it easier for subjects to 
place the cruiser into orbit The number of thrusts required to place the cruiser into 
orbit was taken to be a measure of how well this change informed the subjects of Star 
Cruiser’s speed. Those using the enhanced version required a mean of 3.27 thrusts, 
with a standard deviation of 0.94. The subjects using the original display needed a 
mean of 8.58 thrusts, with a standard deviation of 7.44, to obtain orbit. The data 
contributing to this difference is shown in Figure 7-6. Display [F = 1 1 1.49; p = O.GOOl] 
and Subject [F = 17.91; p = 0.0001] were found to be significant effects after 
performing the analysis of variance on the data. 

Purple Highlight For Cruiser Sneed (Dockingi 

In addition to placing the cruiser into orbit, this display enhancement was 
intended to assist the subject in docking the cruiser. This assistance was accomplished 
again by indicating to the user when the cruiser was at the proper speed. Two 
performance measures, the time to dock and the number of thrusts required to dock, 
were used to evaluate this enhancement. The mean number of thrusts required by those 
subjects using the enhanced display was 1.90 with a standard deviation of 0.72. The 
mean for those using the original display was 2.72 with a standard deviation of 1 .38. 

An analysis of variance performed on the thrust data revealed that Session [F = 2.34; p 
= 0.01 14], Display [F = 34.29; p = 0.0001], and Subject [F = 5.37; p = 0.0001] were all 
significant effects. 

Performing an analysis of variance on the data indicating the time required to 
dock Star Cruiser resulted in similar findings. The same three effects. Session [F = 
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Figure 7-6. Mean Thrusts To Orbit - Noimal Sessions 
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3.87; p = 0.0001], Display [F = 28.91; p = 0.0001], and Subject [F = 3.68; p = 0.0001], 
were all significant Those using the enhanced version had a mean docking time of 4.41 
seconds with a standard deviation of 1.80 while the other group had a mean time of 6.00 
seconds with a standard deviation of 3.01, Figures 7-7 and 7-8 graph, by subject group 
and session, the mean number of thrusts required to dock the cruiser and the mean 
docking time respectively. 

Change Meaning Of The Color Of The Suns 

The purpose behind this enhancement was to inform the users about which 
solar systems contained the most commodities, so users would not waste effort visiting 
those systems where it was not efficient to spend time collecting small amounts of data 
and resources. In order to evaluate this enhancement, the average amount of data and 
resources in all visited and probed solar systems was tallied. Subjects who used the 
enhanced display visited solar systems which contained a mean amount of 629.S units 
of data and resources with a standard deviation of 125.9. Those subjects using the 
original display visited solar systems which contained a mean of 570.1 units of 
commodities with a standard deviation of 132.6. The Session [F = 8.44; p = 0.0001], 
Display [F = 21.28; p = 0.0001], and Subjea [F = 8.94; p = 0.0001] effects were all 
found to be significant upon performing an analysis of variance on this data. The data 
itself is displayed in the graph in Figure 7-9. 

Blue For Enrotv Solar Systems 

To prevent the user from wasting time by sending Star Cruiser into an empty 
solar system, a blue ‘X’ was placed over the sun of each such system in the global map. 
The performance factor used to measure the effectiveness of this display change was the 
number of times the cruiser did enter an empty solar system. Both Session [F = 2. 10; p 
= 0.0232] and Subject [F = 2.18; p = 0.0091] were determined to be significant effects 
from the analysis of variance performed. The data, graphed in Figure- 7- 10, revealed 
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Figure 7-7. Mean Thnists To Dock - Normal Sessions 
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Figure 7-8. Mean Time To Dock (sec,) - Normal Sessions 
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Figure 7-9. Mean Commodities In Visited Solar Systems - Noimal Sessions 
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Figure 7-10. Mean Number Of Empty Solar Systems Visited - Nonnal Sessions 
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that the cruiser visited an empty solar system a mean of 0.385 times with a standard 
deviation of 1.395 if the subject was using an enhanced display. Otherwise, the mean 
was 0.167 with a standard deviation of 0.474. 

Red Highlight For Cmiser Mode 

This display enhancement was designed to indicate to the user what mode, 
deployment of thrusting, Star Cruiser was in at any given time during the task. How 
effective this enhancement was was determined by the number of times the users were 
mistaken (as indicated by their actions) about the cruiser’s nnode. Those subjects who 
were using the enhanced display committed a mean of 0.012 mistakes with a standard 
deviation of 0.046. A mean of 0.044 mistakes with a standard deviation of 0.123 was 
committed by the subjects using the original version of the display. The data for this 
performance measure are displayed in Figure 7-11. The results of performing an 
analysis of variance on the data revealed that all tested effects. Session [F = 2.06; p = 
0.0263], Display [F = 7.51; p = 0.0069], Subject [F = 2.95; p = 0.0(X)4], and the 
interaction of Session and Display [F = 2.03; p = 0.0295], were significant. 

Sessions 15 And 16: With Math Problems 

Subjects’ performance during Sessions 15 and 16, the math sessions, was 
compared to their peifOTmance during Sessions 13 and 14, the last two normal sessions 
performed before the math sessions. Sessions 13 and 14 will also be referred to as the 
premath sessions. Even though the subjects were informed that their scores on the math 
sessions would be adjusted according to their percentage of correct answers to the math 
problems in order to determine who received the bonus money, this adjustment was not 
done for any of the data analyses. When performing the analyses of variance (alpha = 
0.05) on this data, the main effects of interest were Condition (math versus premath), 
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Figure 7-11. Mean Errors In Determining Star Cruiser Mode - Normal Sessions 
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Display type, and Subject. Also of interest were the nested effect of Session within 
Condition and the interaction effect of Condition and Display. 

Overall Performance 

Upon performing an analysis of variance on this data, two of the effects tested, 
EHsplay [F = 5.51; p = 0.0236] and Subject [F = 3.16; p = 0.0016], appeared to be 
significant The interaction effect of Condition and Display was determined not to be 
significant [F = 2.13; p = 0.1514]. The mean score for the subjects using the enhanced 
display on the premath sessions was 13256.3 with a standard deviation of 7194.8. 

When presented with the task of performing math problems simultaneously, the mean 
score for this group increased slightly to 13289.9 with a standard deviation of 4521.2. 
The subjects using the original display, during the premath sessions, had a mean score 
of 12254.8 with a standard deviation of 5023.1. When given the math sessions, the 
original display subjects’ mean score was 8988.0 with a standard deviation of 5559.4. 
These results are displayed in Figure 7-12. The subject group mean scores for the math 
sessions are also plotted next to the scores from Sessions 3 dirough 14 for comparison 
in Figure 7-13. 

Another possible way to measure overall peiforaiance is to monitor the 
number of correct answers to the math problems. Doing so would show that the 
subjects using the enhanced display responded to the math problems with a mean 
percentage of 0.940 correct answers with a standard deviation of 0.049. The other 
subjects had a mean percentage of 0.980 with a standard deviation of 0.019. One 
important factor though, prevents this data from providing a realistic assessment of the 
situation when analyzed. As was witnessed during the actual experiment, there is no 
correlation between percentage of correct answers and the subject’s performance at 
Star Cruiser. For example, a subject may score very high during the cruiser task and 
either get a high or low percentage of correct answers. On the other hand, the subject 
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Mean Display Group Overall Scores - Normal And Math Sessions 
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Figure 7-13. Mean Display Group Overall Scores - Normal And Math Sessions 
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may crash the cruiser early in the session and also either get a high percentage of correct 
answers (since not many problems were presented before the session ended and those 
few were answered correctly) or a low percentage. Therefore, as previously stated, 
analyzing the data associated with the percentage of correct answers to the math 
problems would not help in the determination of which interface is the most beneficial 
with respect to overall task performance. 

Colored Field / Background Colors Of Collection Tools 

The only occurrence of selecting a wrong tool was with the subjects using the 
original display during the premath sessions. A mean of 0.063 errors, with a standard 
deviation of 0.250, was committed. No significant effects were revealed from an 
analysis of variance of this data. 

- Out Too ls When They Cannot Be Deployed 

No significant effects were discovered after performing an analysis of variance 
on the number of selection errors committed by the subjects. The mean number of 
errors committed by the subjects using the enhanced display during the premath 
sessions was 1.188 with a standard deviation of 3.763. These subjects, however, did not 
commit any such errors during the math sessions. On the other hand, those using the 
original display committed a mean of 0. 125 errors with a standard deviation of 0.342 
while ans wering the math problems. The same subjects had a mean of 0.063 errors with 
a standard deviation of 0.250 during the premath sessions. 

Change Color Of Deploy ment / Retrieval String 

An analysis of variance performed on the number of retrieval / deployment 
errors committed revealed that all effects were significant. These effects include 
Condition [F = 99999.99; p = 0.0001], Display [F = 99999.99; p = 0.0001], Subject [F = 
99999.99; p = 0.0001], Session nested within Condition [F = 99999.99; p = 0.0001], 
and the interaction of Condition and Display [F = 99999.99; p = 0.0001]. Subjects 
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using the enhanced display committed a mean of 5.19 errors with a standard deviation 
of 5.96 during the premath sessions and a mean of 4.25 errors with a standard deviation 
of 3.36 during the math sessions. Those using the original display committed a mean of 
19.75 errors with a standard deviation of 16.88 during the pnemath sessions and a mean 
of 15.25 errors with a standard deviation of 15.61 during the math sessions. These 
results are plotted in Figure 7-14. As mentioned. Session nested within Condition was a 
significant effect for this performance measure. The subjects, both groups combined, 
for Sessions 13, 14, 15, and 16, committed a mean of 10.98 errors (standard deviation 
of 13.06), 14.(X) errors (standard deviation of 16.06), 10.94 errors (standard deviation of 
13.85), and 8.56 errors (standard deviation of 1 1.17) respectively. These means are 
charted in Figure 7-15. 

Use Ghost Images 

No effects were found significant after perfomiing an analysis of variance. 
Condition [F = 4.03; p = 0.051 1] and Display (F = 4.03; p = 0.051 1], however, were 
close enough to being significant to warrant mention. The data for this performance 
measure arc pictured in Figure 7-16. The mean number of tools out of commission for 
the enhanced display subjects during the premath sessions to be 0.313 with a standard 
deviation of 0.793. During the math sessions, the mean was 0.688 with a standard 
deviation of 0.793. The performance of subjects using the original version resulted in a 
mean of 0.063 tools out of commission with a standard deviation of 0.250 without the 
math problems and a mean of 0.313 with a standard deviation of 0.602 while answering 
the problems. 

Alter Star Cruiser Capacity Gauges 

The mean percent utilization of Star Cruiser’s capacity by the enhanced display 
subjects was 0.628 with a standard deviation of 0.101 during the premath sessions. 



Mean Number Of Depioyroent/ Retrieval Errors • Pre- 
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Figure 7-14. Mean NutnberOf Deployment / Retrieval Eirors - Pre-Math/Math Condition 



Mean Number Of Deployment / Retrieval Errors (Combined Display Groups) - Pre-Math/Maih Condition 
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Figure 7-15. Mean Number Of Deployment / Retrieval Effors 

(Combined Display Groups) - Pre-Math/Math Condition 



Mean Number Of Tools Oul Of Commission - Pie-Maita/Maih Condiiion 
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Figure 7-16. Mean Number Of Tools Out Of Commission - Pre-Math/Math Condition 
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During the math sessions the mean percentage for this group was 0.557 with a standard 
deviation of 0.133. The subject group using the original display utilized a mean 
percentage of 0.590 of Star Cruiser’s capacity with a standard deviation of 0. 1 84 during 
the premath sessions. The same group of subjects’ mean percentage for the math 
sessions was 0.591 with a standard deviation of 0.205. None of the effects tested during 
the analysis of variance were found to be significant 
White ‘X’ For Ninth Orbit 

The mean time required for the subjects with the enhanced display to place the 
cruiser into orbit was 6.94 seconds with a standard deviation of 2.13 during the premath 
sessions. During the math sessions, the mean time was 8.04 seconds with a standard 
deviation of 2.00. Subjects using the original display required a mean time of 15.99 
seconds with a standard deviation of 12.76 during the premath sessions. While being 
administered the math problems, the original subjects mean time was 15.46 seconds 
with a standard deviation of 6.72. An analysis of variance found that none of the effects 
of question were significant 
Purple Highl ight For Cruiser Speed fOrfaitingl 

An analysis of variance performed on the number of thrusts required for the 
cruiser to be placed into orbit found that Display [F = 46.38; p = 0.0001] and Subject [F 
= 6.21; p = 0.0001] were both significant effects. A mean of 2.85 thrusts with a 
standard deviation of 0.80 was required by subjects participating in the premath 
sessions using the enhanced display. For the math sessions, the same subjects required 
a mean of 3.07 thrusts with a standard deviation of 0.60. Those subjects using the 
original display used a mean of 9.30 thrusts with a standard deviation of 8.88 during 
sessions 13 and 14 and a mean of 8.98 with a standard deviation of 6.08 during the later 
two sessions. These results have been plotted in Figure 7-17. 


Mean Thrusts 'B) Orbit - Pre-Maih/Maih Condition 
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Figure 7-17. Mean Thrusts To Orbit - Pre-Math/Math Condition 
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Purple Highlight For Cru iser Speed fPockine) 

The mean docking time required of the subjects using the enhanced display 
during the premath sessions was 3.91 seconds with a standard deviation of 1.57 while 
during the math sessions it was 4. 1 8 with a standard deviation of 1 .00. The original 
display subjects for the premath sessions required a mean docking time of 4.5 1 seconds 
with a standard deviation of 2.01. For the math sessions, their mean docking time was 
4.57 with a standard deviation of 2.45. No effects were found to be significant after 
performing an analysis of variance on this data. 

The other performance measure for this display enhancement, number of 
thrusts required to dock the cruiser, did have significant effects. Display [F = 4.87; p = 
0.0332] and Subject [F = 2.25; p = 0.0234] were found to be significant after 
performing the necessary analysis of variance. The original display subjects required a 
mean of 2.36 thrusts with a standard deviation of 1.08 for the premath sessions and a 
mean of 2.28 thrusts with a standard deviation of 1.12 for the math sessions. The 
subject group using the enhanced display required a mean of 2.04 thrusts to dock with a 
standard deviation of 0.71 for the pmmath sessions. During the math sessions, they 
required a mean of 1.75 thrusts with a standard deviation of 0.40. This data is graphed 
in Figure 7-18. 

Change Mea ning Of The Color Of The Suns 

An analysis of variance on this performance measure, the average amount of 
conunodities in visited or probed solar systems, revealed that Condition [F = 13.77; p = 
0.0006], Display [F = 20.20; p = 0.0001], Subject [F = 7.57; p = 0.0001], and Session 
nested within Condition [F = 10.85; p = 0.0002] were all significant The mean amount 
of commodities in the solar systems for the enhanced display subjects in the premath 
sessions was 731.7 with a standard deviation of 1 12.7. For the math sessions, the mean 
amount was 618.9 with a standard deviation of 103.6. The subject group using the 


Mean Thnisis To Dock - Pie-Math^ath Condition 
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Figure 7-18. Mean Thrusts To Dock - Pre-Math/Math Condition 
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original display visited solar systems with a mean amount of commodities of 639.4 with 
a standard deviation of 159.2 during the premath sessions and a mean of 545.6 with 
standard deviation of 156.8 for the math sessions. As the subjects performed the task 
using sessions 13, 14, 15, and 16, the mean amounts of commodities contained by the 
visited solar systems in each session was 778.2 (standard deviation of 97.2), 592.9 
(standard deviation of 121.8), 569.3 (standard deviation of 154.9) and 595.3 (standard 
deviation of 1 17.7) respectively. The mean amounts across Condition and Display type 
have been plotted in Figure 7-19. The means across Session have been graphed in 
Figure 7-20. 

Blue For Emntv Solar Systems 

Subjects in neither group ever entered an empty solar system during the math 
sessions. They did, however, do so during the premath sessions. Those subjects using 
the enhanced display entered an empty solar system a mean of 0.500 times with a 
standard deviation of 1.033. The users of the original display entered empty systems a 
mean of 0.438 times with a standard deviation of 0.814. These results are depicted in 
Figure 7-21. An analysis of variance conducted on this performance measure revealed 
that Condition [F = 8.55; p = 0.0055] was a significant effecL 
Red Highlight For Cruiser Mode 

There were only a few occurrence of errors resulting from not knowing what 
nxxie Star C!!ruiser was in during these four sessions. Subjects using the enhanced 
display had a mean of 0.009 errors with a standard deviation of 0.036 during the 
premath sessions and a mean of 0.028 with a standard deviation of 0.017 during the 
math sessions. The subject group using the original display committed a mean of 0.021 
errors with a standard deviation of 0.083 during the premath sessions and, during the 
math sessions, a mean of 0.016 with a standard deviation of 0.063. An analysis of 
variance performed on this data did not reveal any significant effects. 


Mean Commodities In Visited Solar Systems - Pre-Maih^aih Condition 
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Figure 7-19. Mean Commodities In Visited Solar Systems - Pie-Math/Math Condition 



Mean Gxnmodiiies In Visiled Solar Syaems (Combined Display Groups) - Pre-Maih/Maih Condition 
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Figure 7-20. Mean Commodities In Visited Solar Systems 

(Combined Display Groups) - Pre-Math/Math Condition 



Mean Number Of Empty Solar Systems Visited - Pre-Maih/Maih Condition 
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Figure 7-21. Mean Number Of Empty Solar Systems Visited - Pre-Math/Math Condition 
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Sessions 17 And 18 : Transfer To Other Display 

These two sessions had the subjects performing the task as usual with the one 
exception of switching to the other display. As with the sessions involving the math 
problems, subject performance on these sessions were compared to Sessions 13 and 14 
(previously referred to as the “premath” sessions). Since the means and standard 
deviations associated with each performance measure for Sessions 13 and 14 are the 
same as reported in the preceding section, they will not be repeated in the following 
discussion. They are presented, however, in Table 7-2 for reference purposes. As a 
reminder, these are the results obtained while the subjects were using the same display 
as they had been throughout the normal sessions. The data reported in the following 
sections are those that resulted from the subjects switching display types during 
Sessions 17 and 18. The analyses of variance performed on all of this data were used to 
determine the significance (alpha = 0.05) of the effects on performance associated with 
Condition (display used during normal sessions versus display transferred to for these 
sessions). Display type. Subject, Session nested within Condition, and the interaction 
effect between Condition and Display. 

Overall Performance 

When performing the task with the original display, those subjects who 
originally used the enhanced version had a mean score of 15878.6 with a standard 
deviation of 5485.9. The other subject group, having to use the enhanced display for 
these last two sessions, had a mean score of 14559.1 with a standard deviation of 
3331.3. These mean scores are compared to the subject performances on Sessions 13 
and 14 as depicted in Figure 7-22. In addition, the scores obtained for Sessions 17 and 
18 are plotted alongside the scores from die normal sessions in Figure 7-23. When an 
analysis of variance was performed on this data, none 



115 


Table 7-2. Means And Standard Deviations Of Pre-Condition Sessions 
(Sessions 13 and 14) 


Original Enhanced 

Display 


Display 

Enhancement 

Performance 

Measure 

Mean 

Standard 

Deviation 

Mean 

Standard 

PsYiation 

Overall Performance 

Mean Overall Score 

12254.8 

5023.1 

13256.3 

7194.8 

Colored Field / 
Background Colors Of 
Collection Toob 

Select Wrong Toob 

0.063 

0.250 

0.000 

0.000 

‘X’- Out Toob When 
They Cannot Be 
Deployed 

Select Action Errors 

0.063 

0.250 

1.188 

3.763 

Change Coles’ Of 
Deployiitent / Retrieval 
String 

Deployntent / Retrieval 
Errors 

19.75 

16.88 

5.19 

5.96 

Use Ghost Images 

Toob Out Of 
Commission 

0.063 

0.250 

0.313 

0.793 

Alter Star Cruiser 
Capacity Gauges 

Utilization Of Sts 
Cruiser Capacity 

0.590 

0.184 

0.628 

O.lOl 

White ‘X’ For Ninth 
Orbit 

Time To Orbit (sec.) 

15.99 

12.76 

6.94 

2.13 

Purple Highlight For 
Cruiser Speed (Orbiting) 

Thrusts To Orbit 

9.30 

8.88 

2.85 

0.80 

Purple Highlight For 
Cruiser Speed (Docking 
-Time) 

Time To Dock (sec.) 

4.51 

2.01 

3.91 

1.57 

Purple Highlight For 
Cruiser Speed (Docking 
-Thrusb) 

Thrusts To E)ock 

2.36 

1.08 

2.04 

0.71 

Change Meaning Of The 
Color Of The Suns 

Commodities In Vbited 
Solar Systems 

639.4 

159.2 

731.7 

112.7 

Blue ‘X* For Empty 
Solar Systems 

Empty Solar Systems 
Visit^ 

0.438 

0.814 

0.500 

1.033 

Red Highlight For 
Cruiser Mode 

Errors In Determining 
Star Cruiser Mode 

0.021 

0.083 

0.009 

0.036 



Mean Display Group Overall Scores - Pre-Trantfer/Transfer Condition 
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Figure 7-22. Mean Display Group Overall Scores - Rre-Transfer/Transfer Condition 



Mean Di^lay Group Overall Scores - Normal And Transfer Sessions 
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Figure 7-23. Mean Display Group Overall Scores - Normal And Transfer Sessions 
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of the effects were significant at the 0.05 level. The effect of Condition [F = 3.81; p = 
0.0574] was, however, close to being significant. 

Colored Field / Background Colors Of Collection Tools 

None of the effects were significant when the analysis of variance was 
conducted on the performance measure of selecting the wrong tool. Neither subject 
group experienced any errors of this sort after they had switched displays. 

- Out Tools When Thev Cannot Be Deployed 

The analysis of variance also showed that no significant effects existed for the 
performance measure of attempting to select a tool for deployment when it was not 
possible to do so. Few errors of this type occurred. A mean of 0.250 errors with a 
standard deviation of 0.683 was commined by those subjects who were presented the 
original display for the first time. The group which previously had performed the task 
using the original display and transferred to the enhanced version committed a mean of 
0.063 errors with a standard deviation of 0.250. 

Change Colo r Of Deployment / Retrieval String 

The subject group which switched from the enhanced display to the original 
conunitted more deployment / retrieval errors than in the pretransfer sessions while the 
group which switched from the original display to the enhanced version had fewer such 
errors than before. This result is pictured in the graph in Figure 7-24. The group which 
switched to the original version committed a mean of 12. 1 errors with a standard 
deviation of 9.2. The other group had a mean of 1 1.0 errors with a standard deviation 
of 7.5. An analysis of variance revealed that Display [F = 10.31; p = 0.0025] and 
Subject [F = 3.50; p = 0.0006] were significant main effects on this performance 
measiue. The interacdon effect of Condition and Display [F = 13.81; p = 0.0006] was 


also significant. 


Mean Number Of Deployment / Retrieval Errors • Pre-Transfer/IVansfer Condition 
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Figure 7-24. Mean Number Of Deployment / Retrieval Errors - Pie -Transfer/Transfer Condition 
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Ike Ghost Images 

When presented the original display in Sessions 17 and 18, the group which 
had only previously worked with the enhanced version had a mean of 0.313 tools go out 
of commission with a standard deviation of 0.873. The other group, having used the 
enhanced display for the first time in these later sessions, had a mean of 0.250 tools go 
out of commission with a standard deviation of 0.577. No measures, however, as tested 
by the analysis of variance, were found to be significant 
Alter Star Cruiser Capacity Gauges 

The group which switched from the enhanced display to the original had a 
mean percentage of 0.726 with a standard deviation of 0.1 1 1. The other subject group 
which went fiom using the original version to the enhanced had a mean percentage of 
0.692 with a standard deviation of 0.173. The analysis of variance though, did not 
reveal any significant effects. 

White *X* F or Ninth Orbit 

Switching the subject group from the enhanced display to the original resulted 
in their mean time to place the cruiser into orbit equaling 14.58 seconds with a standard 
deviation of 6.29. The other group’s mean time, after being switched to the enhanced 
display, was 9.17 seconds with a standard deviation of 2.62. None of the differences 
resulting firom the effects tested by the analysis of variance were found to be significant. 
Pur ple High light For Cruiser Sneed (Orbitingl 

Having the subject groups switch displays affected the number of thrusts they 
required to place the cruiser into orbit. The subjects which switched to the original 
display increased their mean number of thrusts of 7.57 with a standard deviation of 
3.52. The group using the enhanced display during Sessions 17 and 18 requued fewer 
than before, committing a mean of 3.81 thrusts with a standard deviation of 0.94. An 
analysis of variance on this data showed that the effect of Subject [F = 2.14; p = 0.0258] 
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was significant and the interaction of Condition and Display [F = 22.86; p = 0.0001] 
was significant. 

Purple Highlight For Cruiser Sneed (Docking! 

Those subjects that switched to the original display required a mean 3.55 
seconds with a standard deviation of 1.64. The subjects who switched to the enhanced 
display needed 4.66 seconds to dock with a standard deviation of 1.13. None of the 
effects tested during the analysis of variance proved to be significant. 

The mean number of thrusts to dock the cruiser required by the subject group 
which switched to the original display was 1.74 with a standard deviation of 0.63. 
Those using the enhanced display during Sessions 17 and 18 required a mean of 2.54 
thrusts with a standard deviation of 0.90, a slight increase over the earlier two sessions. 
These changes are graphed in Figure 7-25. The effects of Display [F = 5.59; p = 
0.0230] and Subject [F = 2.25; p = 0.0228] were determined to be significant by the 
analysis of variance. 

Change Me aning Of The Color Of The Suns 

An analysis of variance on the data indicating the average amount of 
commodities present in a visited or probed solar system revealed that Display [F = 7 .38; 
p = 0.0095], Subject [F = 5.48; p = 0.0001], and Session nested within Condition [F = 
8.56; p * 0.0007] were all significant effects. Subjects who switched to the original 
display version visited solar systems with a mean amount of commodities of 


Mean Thnisu To Dock - Pre-Transfer/Tiansfer Condition 
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Figure 7-25. Mean Thnists To Dock - Pre-Transfer/Tiansfer Condition 
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685.5 with a standard deviation of 91.1. Those whose display changed to the enhanced 
version visited solar systems with a mean amount of 662.2 with a standard deviation of 
225.4. The fact that the mean for those using the enhanced version increased while that 
for the other group decreased compared to Sessions 13 and 14 can be seen in the graph 
in Figure 7-26. Figure 7-27 contains a graph showing the change in means over 
session. The mean amounts of commodities in visited solar systems for sessions 13, 14, 
17, and 18 are 778.2 (standard deviation of 97.2), 592.9 (standard deviation of 121.8), 
635.8 (standard deviation of 150.3), and 711.9 (standard deviation of 183.7) 
respectively. 

Blue ‘X* For Empty Solar Systems 

The subject group which switched to the original display visited a mean of 
0.063 empty solar systems with a standard deviation of 0.250. This result is the same as 
for the other group when they switched to the enhanced version. Both groups showed a 
reduction in the amount of empty solar systems visited when compared to Sessions 13 
and 14 as depicted in Figure 7-28. Comparing these sets of results using an analysis of 
variance showed that Condition [F = 5.96; p = 0.0188] was a significant effect. 

Red Highlight For Cruiser Mode 

The analysis of variance did not reveal any significant effects. The subject 
group which switched to the original display from the enhanced committed a mean of 
0.079 errors with a standard deviation of 0. 148. The group which switched to the 
enhanced version from the original committed a mean 0.025 errors with a standard 
deviation of 0.070. 



Mean Commodities In Visited Solar Systems • Pre-Trantfer/Transfer Condition 



Figure 7-26. Mean Commodities In Visited Solar Systems - Pre -Transfer/Transfer Condition 






ilies In Viated Solar Sysiems (Combined Display Groups) - Pre-Transfer/Transfer Cbndition 
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Figure 7-27. Mean Commodities In Visited Solar Systems 

(Combined Display Groups) - Pre-Transfer/Transfer Condition 



Mean Number Of Empty Solar Systems Visited • Pre-Transfer/Transfer Condition 
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Figure 7-28. Mean Number Of Empty Solar Systems Visited - Pre-Transfer/Transfer Condition 
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CHAPTER Vm 
DISCUSSION OF RESULTS 

From the results just presented, it is apparent that the display enhancements 
made to Star Cruiser did indeed have a positive effect on subject performance. This 
conclusion is even true when one considers the effects of Subject variation. Such 
differences between the subjects generally led to high variances in the data within 
subject groups. The individual scenarios also appeared to account for some of the 
variation. Thus, they were not as similar, in respect to difficulty level, as believed when 
creating them. Since, however, each subject was presented the same session at the same 
point in the experiment, any effect caused by scenario feamres would affect all subjects. 
As a result, if Session was the only factor to influence subject performance, then the 
overall scores and individual performance faaors would behave similarly and no other 
effects would be noticed. Since this situation did not occur, other factors must have had 
an effect on performance as well. Display type and Condition (premath versus math 
and first display versus switching displays) are other such factors that could have 
provided such an influence. These two factors contributed to many significant effects 
as discovered by the analyses of variance that were performed and previously discussed. 

For the most part, the display enhancements in^jroved subject performance. In 
some cases, though, that no differences were found and in other instances the opposite 
of what was expected occurred. The following discussion attempts to explain the 
implications of these differences, or absence of differences, in performance. 



128 


During Sessions 15 and 16, the subjects encountered the additional task of 
having to perform simple arithmetic problems. The question is not whether or not 
performance suffers, but which display group’s performance is affected the most by the 
added task. Performance is expected to suffer for both groups simply because another 
task was added to an already complex one. It was believed that the enhanced subjects’ 
performance would not suffer as greatly as the original display group since the display 
enhancements would allow those subjects to devote more cognitive resources to 
answering the math problems without greatly sacrificing any used to perform the Star 
Cruiser task. Below, the subject groups’ overall performance is compared as well as 
the results from the individual performance measures. Data for Sessions 15 and 16 are 
compared between display groups to determine if the addition of the math problems 
affected the groups differently and, if so, how. The same data for each group is then 
compared to Sessions 13 and 14 to determine what type of effect the added task had on 
performance. 

Sessions 17 and 18 involved both subject groups transferring between display 
types. As with the math sessions, overall and individual performance measures were 
compared to Sessions 13 and 14. This comparison was done in order to determine how 
well the subjects’ performances transfer to a display with which they are not familiar. 
Performance was expected to decrease for those who switched to the original display 
version. Performance for the other subject group was expected to remain constant or 
increase upon switching to the enhanced version. Of concern is whether the sudden 
absence or presence of certain perceptual cues will affect subject performance. In the 
following, the Condition effect refers to any differences (Pre-Math/Math; Pre- 
Transfer/Transfer) between the sessions being analyzed. 
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Overall Performance 
Normal Sess ions: Sessions 3 Through 14 

It was originally thought that those subjects using the enhanced display would 
perform, as measured by score, better at the Star Cruiser task than those using the 
original display. From the data, it appears that this hypothesis was confirmed. Display 
type was shown to be a significant effect in the subjects’ ability to score well. Session 
was also found to be significant, indicating a possible learning effect and also the 
possibility that the sessions varied in difficulty. The Subject effect was also determined 
to be significant, as is typically the case in studies of complex human performance. At 
a general level, however, one can say with some confidence that the display 
enhancements as proposed by the ecological task analysis helped contribute to better 
overall task performance. How these changes aided the users are addressed by the 
individual performance measures discussed shortly. 

Also of question was how these display enhancements would affect learning to 
perform the task. By studying the trend of scores during Sessions 3 through 14 for both 
subject groups, an important insight can be gained. This insight is that the subject 
group using the enhanced display began the sessions inunediately after training already 
scoring higher than the other group. This result suggests that the display enhancements 
were a factor in how well the subjects initially learned the task. 

With only two exceptions, the mean scores for both groups increase and 
decrease together depending on the session. The scores are approximately the same 
starting at Session 1 1 with the original display subject group actually having a higher 
mean score than the enhanced group on Session 11. Given this result and the fact that 
there is a maximum score that can be obtained (though given the dme limit on the 
session, this score was unattainable), one is led to believe that those using the enhanced 
display were able to reach this maximum potential sooner than the other subjects. After 
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a while though, the subjects using the original display were able to match the 
performance of the enhanced display users. Thus, it seems that the enhanced display 
subject group was able to learn the Star Cruiser task, not necessarily better than, but 
more quickly than the original display subject group. 

Math Sessio ns: Sessions 15 And 16 

The data for the subjects’ scores during the math sessions show that the 
performance of the subjects using the enhanced display remained fairly constant while 
the performance of the group using the original display decreased. The Condition effect 
(math versus premath) was found to be not significant This result makes sense since 
otherwise both subject groups would have seen their performance worsen. The Subject 
effect however, was found to be significant This result reflects the large variance in 
means scores that exists within subject groups. The Display effect, which was found to 
be significant accounts for the differences in mean scores of the two groups. This 
result is expected as similar results discussed for the Normal sessions revealed that 
differences in overall performance do exist between display types. Since the interaction 
effect of Condition and Display was not significant the extent to which the enhanced 
display allowed the subjects to better reallocate their resources for performing both 
tasks is questionable. Though overall performance may not have differed significantly 
due to the interacti<Mi effect between the display types and the introduction of a second 
performance along the individual performance measures may have. 
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Transfer Sessions: Sessio ns 17 And 18 

Upon switching display versions, both subject groups experienced increases in 
their scores. In addition, the Condition effect was found to be nearly significant [F = 
3.81; p = 0.0574]. This outcome indicates that the increase in mean scores for both 
subject groups was due to switching displays. The fact that subject performance 
improved for those who switched to the enhanced version was not a surprise. It was 
believed that, given the additional perceptual cues and the information they provide, the 
subjects would discover the task to be easier than while using the original version. It 
was quite surprising, however, to find that the subjects who originally used the 
enhanced display also improved their performance scores after switching to the original 
display. 

Two possible explanations exist for why the enhanced subjects performed just 
as well on the task using the original display. The first is that because they were 
presented something new, in this case in the form of a different display with certain 
perceptual cues lacking, the subjects concentrated more on performing the task. This 
concentrated effort may have compensated more than enough for the loss of information 
due to the missing cues and, as a result, task performance was as high or higher than 
before. This rationale may also be adapted to apply to those subjects who switched to 
the enhanced display and performed better. The second possible explanation is that the 
display enhancements allowed the subjects to learn more about the Star Cruiser task 
than what was presented at the display surface level. As a result, this added knowledge 
transferred with the subjects as they switched screens. Since the subjects had learned 
from the display enhancements how to acquire this embedded information from the 
task, independent of the cues themselves, they were able to continue to perform the task 
well, and in some cases, better. 
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Individual Performance Measures 

Each perfomance measure that was monitored during the experiment was 
associated with one or more of the display enhancements. Each enhancement is 
discussed in terms of how it affected performance during the normal, math, and transfer 
sessions. The effect each display enhancement had on performance is considered in the 
discussion of the data from the normal sessions. The following discussion is also 
concerned with how the additional task requirements of answering math problems 
affected the effects of the display enhancements. In other words, would the arithmetic 
demands affect subject performance equally, if at all, regardless of display type? Or 
would it affect one subject group more than the other? Each display enhancement’s 
individual performance measure was also analyzed to determine if switching displays 
had any effect on subject performance. This analysis would allow one to determine 
which measures were affected and how they were afrected which would then allow one 
to conceptualize, with a little more reliability, a thewy to explain why overall 
performance improved for both subject groups during the transfer sessions. 

Colored Fiel d / Background Colors Of Collection Tools 

The intent behind diis change was to assist the user in determining the 
functions and limitati ons of the collection tools. As seen in the data, during the normal 
and transfer sessions, only one subject selected the wrong type of tool. Therefore, it 
appears that this enhancement did not have any effect on subject performance during 
Sessions 3 through 14 or during the transfer sessions. In addition, this display 
enhancement did not have an affect during the math sessions. 

It would seem that this particular enhancement would have its biggest 
influence during the eaiiier sessions. This enhancement may have been one of the 
reasons why the enhanced display group was able to perform so much better than the 
original display subjects right after completing the training. The addition of math 
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probiems may also affect performance on this measure during training. Switching the 
display versions also may have possibly had a bigger impact on performance if it 
occurred during training since training is where these errors arc most likely to be 
common. Unfortunately, since none of the data for the first two days was analyzed, this 
theory is largely speculation. 

•X*-Qut Too ls When They Cannot Be Deployed 

This enhancement did not seem to have a major effect on the users’ 
performance during any of the sessions. Similar to the previous display change, 
however, this change would also seem to have the nx>st potential benefit during early 
sessions since its primary purpose was to inform the user of the situations when a tool 
could not be deployed- After a while, these instances should be learned by the subjects 
and thus, selection errors should not occur as often. This enhancement may have also 
been a contributing factor to the enhanced display subjects increased performance right 
after the training sessions. 

Change Color Of Deplo yment / Retrieval String 

According to the data, during Sessions 3 through 14, those subjects who used 
the enhanced display were better able to deploy and retrieve tools than the other 
subjects. In addition, both Subject and Session effects were found to be significant on 
this measure. The differences in the mean number of deployment / retrieval errors 
committed by each subject group can be attributed to the differences in displays. As 
reported, the Display effect was significant. By observing the subjects during the 
experiment, it became obvious that this display change provided subjects the necessary 
infemnation for determining when to release the mouse button to complete the 
deployment / retrieval action. In addition, since the difference in means appeared to 
remain relatively constant, this tracking task was not one that improved quickly over 
time. Consequently, since a subject’s ability to perform this task can greatly influence 
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not only the collection of the commodities but how much time is remaining to do other 
tasks, it is fair to say that this display enhancement contributed to the differences in 
overall performance exhibited by the two groups during the normal sessions. 

As discussed in the ecological task analysis, subjects considered the tasks of 
deploying and retrieving tools as two of the most difficult Thus, the addition of the 
math problems would lead one to believe that an increase in deployment and retrieval 
errors would result. The opposite, however, occurred. Both subject groups reduced 
their mean number of errors committed during the math sessions. This result could 
have been simply accounted for by a learning effect and improved abilities at 
performing this action except that all effects were determined to be significant The fact 
that the Session effect nested within the Condition effect was significant indicates that 
the differences between the session scenarios, regardless of whether or not the sessions 
involved the math problems, attributed to some of the variation in the number of errors 
committed. The Display effect would reflect the differences between the means of both 
subject groups. The fact that Condition was significant indicates that the presentation 
of math problems to the subject groups did have an influence on their performance. The 
interaction effect between Condition and Display would signify that this effect varied 
depending on the display type. 

The effect of Condition, contrary to earlier conjecture, improved the subjects’ 
ability at performing this task. It appeared to have improved the original display 
subjects* performance to a slightly greater degree than the enhanced display group. A 
possible explanation for this result is that the subjects, faced with the even more 
dif ficult challenge of having to perform this action while answering math problems, 
simply tried harder, paid nwre attention, and/or took their time in performing the 
deployment and retrieval actions. Or possibly, the Star Cruiser task can be performed 
better while not thinking about it, like quickly running down the stairs. In any case, the 
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result would be fewer errors committed by the subjects. One can still state with 
confidence that this display enhancement did in fact contribute to improved overall 
performance even with the presence of the additional task of answering math problems. 
Whether or not an additional task would force subjects to try harder at this action in the 
future, and thus perhaps increase their overall score, would depend on the motivation of 
the subjects and on the difficulty of the additional task. 

The mean number of deployment and retrieval errors committed by the 
subjects who switched from the enhanced display to the original display increased. The 
mean number decreased for those who transferred from the original to the enhanced 
display. The Display effect was also significant and would account for the large 
differences in the means between each subject group. The interaction effect of 
Condition and Display, which was significant, indicates the fact that switching displays 
had a definite effect on this performance measure. As was expected, those subjects who 
switched to the original display version, since they lost the use of this perceptual cue, 
committed significantly more deployment / retrieval errors than before they made the 
change. The other subject group, which transferred from the original to the enhanced 
display, co mmi tted significantly fewer errors than before since they now had the benefit 
of the perceptual cue. These results would help explain why the subject groyp which 
went from the original to the enhanced display were able to improve their overall 
pe rform ance. The fact though, that the other group’s performance on this measure 
worsened indicates that this enhancement was not responsible for those subjects’ 
improved overall performance. 
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Ike Ghost Images 

Both subject groups seemed to be equally likely to render a tool out of 
commission while performing the Star Cruiser task during the normal sessions. As a 
result, neither the Subject nor Display effects were found to be significant. The Session 
effect was, though, determined to be significant This result is not surprising. The 
likelihood of overloading the Star Cruiser seemed to be dependent on the distribution of 
Hata and resources to the various solar systems within the galaxy. If a galaxy contained 
any systems which possessed enough conunodities to overload the cruiser, than it would 
increase the subjects’ chances of placing a tool out of commission. Since each galaxy 
varied with each scenario and session, the opportunity existed for variation in subject 
pcrf(wmance on this measure. As a result, high variation in the subject data occurred 
and the effect due to Session was found to be significant 

One surprising result was that the enhanced display group allowed more tools 
to go out of commission than the original display group during the math sessions. The 
difference between the group means can be accounted fw by the nearly significant 
effects of Display [F = 4.03; p = 0.051 11 and Condition [F = 4.03; p = 0.051 1]. With 
regard to Condition, it is obvious from the data that performing the math problems 
hampered subjects’ abilities to determine properly when the data or resources collected 
by a tool would overload the cruiser upon retrieval. It also appears that those subjects 
using the original display were better able to make this determination than those using 
the enhanced display, thus explaining the significance of the Display effect 

It may, however, not be the case that the original display was better suited for 
this performance measure, but rather the enhanced display simply suffered to a greater 
extent due to the additional task. This conclusion is supported by the fact that during 
the normal sessions there was no significant difference between the two displays. 
Though speculative, a possible explanation for these results is that, due to having to 
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answer the math problems, the subjects using the enhanced display did not attend to the 
information this enhancement provided. Perhaps they felt so comfortable enough in 
performing the action that they could ignore the cue. Thus, they became overconfident 
in their abilities to do the action and they committed more mistakes. This behavior is in 
contrast to the original display group which never had the perceptual cue in the first 
place. These subjects did not have the additional infoimadon with which to ignore and, 
therefore, proceeded to perform this action as they would normally do. As a result, this 
performance measure did not suffer as gready for the original display subjects as for the 
enhanced version subjects. It is cases such as this one where the introduction of an 
additional task might result in the display enhancement hindering overall performance. 

Switching display versions did not have any apparent effect on the subjects’ 
abilities to prevent tools from going out of commission. Furthermore, the analysis of 
variance failed to find any factors which were significant It is as if the subjects who 
lost the use of the perceptual cue were able to remember what data and resources had 
been collected and those which gained the cue never truly relied on it This data 
therefore suggests that this cue was not a factor in determining overall performance. 
Alter Star Cruiser Capacity Gauges 

This enhancement was designed to bring to the display surface information 
regarding the current exact percent utilization of Star Cruiser’s capacity. Before 
enhancement it was often difficult to determine utilization due to the differences 
between the pie graphs on the planets and the bar graphs used to indicate capacity. The 
Hata from the nor mal sessions suggest that this display enhancement resulted in subjects 
docking at Star Base with a higher percent utilization of Star Cruiser’s storage capacity 
than those subjects using the original display. Both Subject and Session were also 
revealed to be significant effects on this measure. This finding explains the 
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considerable variation that existed within subject group means. Display type was also 
significant as was the interaction between Session and Display. 

The fact that the interaction effect was so significant for these sessions would 
indicate that possibly the effectiveness of the display type was dependent on the session 
configuration or the different subjects’ ability to learn over the sessions how to use the 
gauges. As previously mentioned, each scenario varied in the amount of commodities 
present in each solar system. Therefore, some scenarios made it slightly difficult for the 
subjects to fill the cruiser to near capacity. For those particular sessions then, the 
enhanced display subjects performed better along this measure. For those sessions 
where it was very simple or not the best option to load the cruiser to near capacity, than 
no differences would have existed. Those sessions where the original subject group had 
a higher percent utilization than the enhanced display subjects would be indicative of 
these types of sessions. Since this situation occurred but a few times (Sessions 3 and 8), 
some of the performance differences along this measure can be attributed to variation in 
displays alone. Furthermore, since overall performance is measured by how many 
conunodities are returned to Star Base, the higher the percent utilization the better. 
Hence, the enhanced gauge display did contribute to the subject groups’ differences in 
overall performance during the normal sessions. 

The p erform ance measure associated with this display enhancement did not 
show any significant effects, for either group of subjects, as a result of the math 
sessions. Therefore, it seems as if the effectiveness of this display change was not 
affected by the addition of the math problems as it neither assisted nor hindered the 
subjects during these sessions. As a result, the overall perfcwmances of the subjects was 
not influenced. 

No sigitificant effects were found by conducting an analysis of variance on the 
performance data for this enhancement during the transfer sessions. The absence of an 
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effect though, lends support to one of the explanations of why the enhanced-to-original 
display subject group did not show a decrease in overall performance. It is possible 
that, over time, these subjects were able to learn exactly how much data and resources 
Star Cruiser could carry. This theory is quite reasonable since the perceptual cue for 
this performance measure in the enhanced version was essentially a number of empty 
planets representing the total cruiser capacity. Since they had learned what the exact 
capacity was and since the sudden absence of this cue did not mean that the capacity 
changed, these subjects were able to perform the task using the original display as they 
had been before with the enhanced display. Thus, performance remained fairly 
consistent across session types. 

White For Ninth Orbit 

Hidden within the depth structure of the original display are clues that can be 
used by the subject to locate the ninth orbit The use of the white ‘X’ was intended to 
inform the subject of the location clearly and accurately. The data suggests that this 
display enhancement accomplished just that Never once did the original display 
subject group have an overall mean orbiting time less than the enhanced display group 
during the normal sessions. In addition, the effect of Session was determined to be 
significant while that of Subject was significant Display was also significant which 
would account for the large differences in mean times between the two groups of 
subjects. This enhancement contributed to the enhanced display subjects’ increased 
overall performance by locating the proper orbit which essentially gave them more time 
to perform other aspects of the Star Cruiser task. Whether or not it was easier for 
those subjects using the enhanced display during the normal sessions to place the 
cruiser into orbit cannot be determined without discussing how proficient the subjects 
were at controlling the cruiser’s speed in order to place it into orbit. 
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In addition, no significant effects were found when analyzing the subjects’ 
mean times to orbit during the math sessions. Consequently, it appears as if the added 
task of the math problems did not result in any differences between the displays or 
between sessions 13 through 16. Asa result, one may state that the additional cognitive 
task did not affect this display enhancement’s effectiveness in any way and, therefore, 
did not affect overall performance. 

Though the enhanced display subjects saw their mean time to orbit increase 
upon switching to the original display during the transfer sessions, and though the times 
for when the original display subjects switched displays decreased, no significant 
effects were found. These results suggest that the enhanced display subjects were able 
to learn how to extract the information, normally provided by the perceptual cue, from 
the depth structure of the display without having to rely on the cue itself. Therefore, 
one can not state with any certainty that final subject performance on each display type 
differed due to the loss ot gain of perceptual cues and the information they conveyed. 
Purple High light For Cruiser Sneed (Orbiting) 

The only way users were able to determine Star Cruiser’s speed was by 
associating speed with how quickly it transverses the screen. The purple highlight ring 
was designed to let the subject know when the cruiser’s speed had reached the 
maximum allowed for obtaining orbit. Counting the number of thrusts the subjects 
needed to apply beftwe the cruiser entered orbit revealed how well they were aware of 
the cruiser’s speed. The data from the normal sessions shows that those using the 
enhanced version performed fewer thrusts than the subjects using the original version. 
Though this finding may be partially attributable to being able tt> locate the ninth orbit 
more easily (as previously discussed), it is also due to having the information 
concerning the cruiser’s speed displayed. 
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The effect of Subject differences was determined to be significant. Though not 
much variation in the number of thrusts applied was present in the enhanced subject 
group, the extreme amount in the other group was enough to cause this effect to be so 
significant. The fact that Display was a significant effect supports the claim that this 
enhancement proved useful in improving subject performance during these sessions 
along this measure. Since the enhanced display provided for better subject performance 
along both measures, cruiser speed and the location of the ninth orbit, which must be 
considered to place the cruiser into orbit, it is reasonable to state that the subjects using 
the enhanced display were better able to place the cruiser into orbit compared to the 
other subject group. Therefore, witiiout a doubt, these last two display enhancements 
contributed significantly to the increase in the subjects’ overall performance of Star 
Cruiser during normal sessions. 

Similarly, both Subject and EHsplay effects were significant for the math 
sessions. This result is still indicative of the large variation within the subject groups as 
well as the difference in mean thrusts required few orbiting between the two groups. 

The fact that Condition was not significant indicates that the extra effort required to 
perform the additional math task did not effect this display enhancement’s 
effectiveness. Even during these sessions the display change helped to improve overall 
performance. 

The fact that the Subject effect was significant for this performance measure 
during the transfer sessions accounts few the relatively high variation that exists within 
the subject groups in the mean number of thrusts required to place Star Cruiser into 
orbit As was expected, the enhanced-to-original display subject group experienced an 
increase in their mean number of required thrusts. The original-to-enhanced display 
group required fewer thrusts. Not only was this outcome so, but when subjects, 
regardless of subject group, used the enhanced display, they would require fewer thrusts 
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to orbit the cruiser. This finding explains why the interaction effect of Condition and 
Display was significant on this measure. No effects, however, were found significant 
for the other factor, the time to place the cruiser into orbit. Hence, whether or not the 
enhancement improves overall performance based on a subject’s ability to place the 
cruiser into orbit is unknown for when the users switch display types. 

Purple Highlight For Cruiser Sneed (Docking) 

The purple highlight enhancement was also intended to improve a user’s 
ability to dock Star Cruiser at Star Base. As with placing the cruiser into orbit, both the 
time and the number of thrusts required to complete the action are important in 
determining the effectiveness of this display change. As the graph of the data from the 
normal sessions showed, the subject group using the enhanced display required less 
mean rime to dock the cruiser for every session but one (Session 5). The analysis of 
variance for the amount of time required to dock the cruiser during these normal 
sessions revealed that Session, Subject, and Display were all significant effects. Once 
again. Subject and Session differences explain the large variances in the times required, 
especially for the group of subjects using the original display. The difference in mean 
times required can be attributed to the Display factor. 

Having to answer math problems while performing this activity, as measured 
by the rime required to dock, did not have any significant effect on the subjects’ 
abilities to do so. Nor was any one subject group better able to perform the action than 
the other. As a result, the additional math task did not affect this enhancement’s 
assistance to the user in docking Star Cruiser. Switching display types also had no 
effect on the amount of time the subjects required to dock the cruiser at Star Base. The 
analysis of variance failed to identify any factors which significantly affected 
performance of this action. Therefore, it is also safe to state that this display 
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enhancement did not aid or hinder the subjects upon switching display versions in the 
time required to dock the cruiser. 

The graph depicting the number of thrusts required to dock the cruiser during 
the ntnmal sessions also shows that only during Session S did the subject group using 
the original display have a lower mean. The effects of Subject, which was significant, 
and Session, which was significant, account for the large differences in number of 
thrusts required within subject groups. The significant Display effect accounts for the 
difference in the means between the two groups. Both the mean time and thrusts 
required for docking were significantly lower for the subject group using the enhanced 
display compared to the group using the original version during the normal sessions. 
Thus, as with placing the cruiser into orbit, it appears as if the display enhancement of 
highlighting the cruiser when it has reached the proper docking speed, by bringing to 
the display surface information regarding cruiser speed, allowed subjects to better 
perform this action, at least during the normal sessions. Being able to better perform 
this action then allowed the enhanced subjects more time to perfcnm other tasks within 
Star Cruiser and therefore improve their overall performance over the original display 
subjects. 

Both Subject and Display were also found to have significantly effected the 
subjects’ abilities to dock the cruiser as measured by the number of thrusts required 
during the math sessions. As before, the Subject effect is noticeable in the large 
variation within subject groups. The Display effect is seen in the enhanced display 
subjects requiring fewer thrusts to dock the cruiser than the original display subjects. 
Again, since the Condition effect was not significant, the math problems did not affect 
the subjects’ performances along this measure. Whether or not the display 
enhancements supporting the activity of docking the cruiser are effective or not during 
these sessions cannot be concluded since no significant Display effect existed for the 
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rime required to dock. It can be stated, though, with confidence that the additional math 
task did not interfere with any of the enhancement’s effectiveness. 

The number of thrusts required to dock the cruiser during the transfer sessions 
also resulted in the Display and Subject effects being significant Those subjects who 
switched from the enhanced display to the original required fewer thrusts to dock the 
cruiser during Sessions 17 and 18 than during Sessions 13 and 14. The other subject 
group required slightly less as well. The high vaiiarion in mean thrusts required within 
each subject group is responsible for the significance of the Subject effect The Display 
effect is caused by the difference in the means between the two types of displays. The 
fact that the original-to-enhanced display subject group had better performance can 
possibly be attributed to the gain of the information provided by the perceptual cue. 
More than likely though, it is for the same reason that the enhanced-to-original group’s 
performance also improved. Both groups, by this time, had been so well-practiced at 
the task of docking the cruiser that the presence or absence of the cue and its 
information did not matter. Therefore, at least after the subject has become highly 
skilled at the task, the highlighting of the cruiser for docking purposes has little, if any, 
effect on the performance of this action or, for that matter, overall task performance. 
Change Meaning Of The Color Of Suns 

It was thought that if users were better informed as to which solar systems 
contained the most commodities compared to the others, then they would be able to 
send the cruiser to them and thus maximize their collection efficiency. This display 
enhancement was intended to make the information concerning the solar systems’ 
contents clearer and more accurate. The change seemed to have helped during the 
normal sessions as the enhanced display subjects visited and probed solar systems 
which contained, on average, more commodities than those visited and probed by the 
original display subjects. This result occurred during every normal session except for 
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Session 4. Once again, there was relatively high variation in the means within the 
subject groups. This result correlates with both the Subject and Session effects being 
significant. The extreme significance of the Display effect helps to explain the 
differences between the subject group means. This finding suggests that this display 
enhancement allowed the subjects to better determine which solar systems contained the 
most data and resources. As a result, the enhancement also helped to improve overall 
task performance since the subjects were not only able to collect more commodities 
while visiting fewer solar systems, but were also better able to determine which systems 
would fill the cruiser to near maximum capacity. 

The effects of Condition, Display. Subject, and Session nested within 
Condition on which solar systems the subjects decided to visit or probe during the math 
sessions were all determined to be significant. Once again, the Subject and Session 
effects account for the relative high variances in the performance measures within the 
subject groups. The Display effect signifies the differences that exist between the 
subject groups. The Condition effect is indicative of the additional math task having an 
influence on the subjects’ performance. The ability of each subject group to visit or 
probe those systems containing the most data and resources suffered as a result of 
having to answer the math problems. The fact that the enhanced display group still 
perfonooed this activity better than the other group suggests that even though they were 
required to pe rform an additional task, it did not affect this enhancement’s effectiveness 
in helping the enhanced version subjects achieve a higher overall performance for these 
sessions. 

Depending on how the galaxy and its solar systems in each session are 
configured, subject performance at visiting or probing those which contain the most 
Hafa and resources will vary. Thus, as was found, during the transfer sessions, the effect 
of Session nested within Condition was significant. The analysis of variance performed 
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on this performance measure also revealed the Subject and Display effects to be 
significant. As the data revealed, subjects who switched to the enhanced version were 
then more likely to visit those solar systems which contained the most commodities 
than when they were using the original display. On the other hand, those who switched 
to the original display from the enhanced version became less likely to visit those 
systems more plentiful in data and resources. 

The information this cue provides cannot be learned by the subjects since the 
actual meaning of it changes with display type. This reasoning would explain the 
worsened performance along this factor for those subjects who switched to the original 
display. Hence, the perceptual cue of the color of the sun and, most importantly, the 
meaning associated with it, were an influence in determining where the subjects sent the 
cruiser. Since the amount of data and resources they will be able to return to Star Base 
is dependent on where they steer Star Cruiser to, this display enhancement has an effect 
on the subjects’ overall performance. It may be concluded, however, that this 
enhancement does not have a great impact on overall performance as demonstrated by 
the enhanced-to-original subject group’s ability to still improve their overall 
performance. 

Blue ’X’ For Empty Solar Systems 

This display enhancement was also intended to assist the user in determining 
where, and where not, to send the cruiser. The original display does not provide the 
user with any information concerning whether or not a solar system is completely 
empty (it contains no tools or commodities). Anytime the cruiser enters one of these 
systems, it only wastes time and prevents the user from performing other actions which 
may be more beneHcial. By placing an ‘X’ over the sun to indicate empty solar systems 
in the enhanced display, it was expected that the subject group using that display would 
visit fewer empty solar systems than the group using the original display. The data 
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from the normal sessions, however, revealed no significant differences existed between 
the two display types, as this situation did not occur very frequently. The analysis of 
variance did though, find Session to be significant This finding makes sense since the 
different galaxy scenarios would make some sessions more likely to contain empty 
solar systems than others, thus increasing the chance of the subjects entering one. The 
effect due to Subject was also found to be significant The high variations within 
subject groups on this performance measure are explained by the significance of these 
two effects. Since Display type was not significant however, it cannot be stated that 
this enhancement helped improve the enhanced display subjects’ overall performance. 

Neither subject group had Star Cruiser enter an empty solar system during the 
math sessions. Considering that during Sessions 13 and 14 subjects in both groups did 
so. Condition was a significant effect This result is most likely due to one of several 
reasons. The solar system scenarios may have been created so that there were fewer 
opportunities for the subjects to enter empty systems during the transfer sessions. Even 
though the Session nested within Condition effect was not found to be significant, this 
difference between the scenarios could explain the differences in the data. Another, 
more plausible explanation is that the criteria judged by subjects when determining 
whether to perform this task, combined with the differences in session scenarios, did not 
warrant entering empty solar systems. Finally, the subjects’ may have learned to avoid 
the empty systems. Whichever applies here, the fact remains that since there were no 
significant effects from the differences in the display types, this display enhancement is 
not affected by the additional task. 

The only effect found significant regarding this enhancement during the 
transfer sessions was Condition. Subjects visited more empty solar systems in the 
sessions before they switched displays. Differences in the scenarios, subject methods 
for perfor min g the task, and/or subject learning may have contributed to the subjects 
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visiting fewer empty solar systems. No matter the case though, this display 
enhancement does not seem to affect the subjects’ performances of this factor, or 
overall performance therefore, as they become more experienced at the task. 

Red Highlight For Cruiser Mode 

Users often do not realize what effect a string pull will have on Star Cruiser. 
Often, if a user fails to select the cruiser while trying to deploy a tool, the user will not 
realize that a string pull performed inunediately will apply a thrust to the cruiser rather 
than deploy a tool. This problem arises because the original display presents no cues 
which inform the users of Star Cruiser’s mode. The red highlight ring was designed to 
provide this information. The effectiveness of this enhancement was measured by 
counting the number of errors the subjects committed because they were unaware of 
what the cruiser’s mode was. Therefore, those subjects using the enhanced display 
should have committed fewer such errors. 

Though few of these errors did actually occur during the normal sessions, the 
graphed data clearly shows that subjects using the enhanced display only once (Session 
6) committed more errors than the other subjects. The relatively large variances within 
the subject groups can be attributed to the significant effect of Session and the 
significant Subject effect The differences in means between the two groups are 
accounted for by the Display effect which was significant and the significant 
interaction effect between Display and Session. The fact that the interaction effect was 
also significant is a possible indication that the Display effect varied depending on the 
Session. One can state with confidence that, during the normal sessions, the red 
highlight enhancement prevented the subjects from committing more time costly errors, 
errors which could also result in the cruiser crashing into the sun, compared to the other 
subject group and thus helped to bener their overall performance. 
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Requiring the subjects to answer math problems in addition to performing the 
Star Cruiser task did not appear to have any effect on their ability to determine the 
mode of Star Cruiser. In addition, the performance of those subjects using the enhanced 
display did not seem to differ from those of the original version. This finding is 
supported by the absence of any significant effects during the math sessions. Therefore, 
one may conclude that the performance of the additional task does not interfere with the 
effectiveness of this display enhancement in contributing to overall performance as 
demonstrated in the normal sessions. 

An analysis of variance on the number of errors committed due to not realizing 
the cruiser mode during the transfer sessions did not indicate any significant effects. It 
appears then that, over time, the subjects became better able at determining the cruiser’s 
mode to the point where the information provided by the cue was not necessary. Thus, 
this display enhancement did not affect overall performance during these later sessions. 

Recap Of Results And Discussion 

After determining the effects each of the twelve display enhancements had on 
overall performance, some interesting conclusions can be drawn. First, though, it must 
be mentioned that the display changes can be divided into two groups; those which 
affect the user’s choice of action and those which affect the manner in which an action 
is performed. In the following, the former are referred to as action-selection 
enhancements while the latter will be considered as action-performance enhancements. 
The enhancements are listed in Table 8-1 according to which type they are. 

User performance is definitely affected by the display enhancements as the 
results have demonstrated. During the normal sessions, all five of the action- 
performance enhancements contributed to improving the enhanced display group s 
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Table 8-1. Action- Selection And Action-Performance Enhancements 

Action-Selection Enhancements 
Colored Field 

Background Colors Of Collection Tools 

‘X’ - Out Tools When They Cannot Be Deployed 

Use Ghost Images 

Alter Star Cruiser Capacity Gauges 

Change Meaning Of The Color Of The Suns 

Blue ‘X’ For Empty Solar Systems 

Red Highlight For Cruiser Mode 

Action-Performance Enhancements 
White ‘X’ For Ninth Orbit 
Change CoIot Of Deployment / Retrieval String 
Purple Highlight For Cmiser Speed (Orbiting) 

Purple Highlight For Cruiser Speed (Docking - Time) 
Purple Highlight For Cruiser Speed (Docking - Thrusts) 
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performance. In addition, two of the action-selection enhancements (Star Cruiser 
Capacity Gauges and Color Of The Suns) also helped to improve performance. Three 
other action-selection enhancements (Colored Reid, Background Colors, ‘X’-Out 
Tools) were speculated to be more beneflcial during the training sessions. Thus, it 
appears that, with regards to normal task performance, those enhancements which aid 
the user in action-performance will contribute to overall improvements in performance. 
Those enhancements which help the user to decide which possible action to perform, 
though some apparently influenced the subjects’ performances during the normal 
sessions, mainly help the user, if at all, during training. This theory seems logical since 
rules and constraints governing action-selection usually occurs during these sessions. 

During the math sessions, only half of the enhancements influenced subjects 
performances. Of those, only two enhancements were not affected by the presence of 
the math problems. The effectiveness of the enhancements involving the purple 
highlight for orbiting and docking, both designed to increase skill acquisition, was not 
influenced by the subjects’ reallocating their cognitive resources to answer the math 
problems. The other four enhancements were, however, affected. It seems as though 
those actions involving these changes (Ghost Images, Sun CoIot, Blue ‘X’, String 
Color) demanded more cognitive resources than the subjects’ were able to devote to 
them w hil e performing the math sessions. The fact that three of these enhancements 
were involved with action-selection (Ghost Images, Sun Color, Blue ‘X’) is not 
surprising since those type would generally require the additional mental attention of 
the user more often than a action-performance enhancement Though the other 
enhancement was to improve the subjects’ performance at the action (String Color), 
since it was rated as the most difficult actions during the task analysis, it would seem 
that it would require a greater deal of mental resources than the other actions. 
Therefore, performance would suffer more. 
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When subjects performing a complex task are presented with another non- 
menial task, those activities which compete for similar resources, in this case mental, 
will suffer (Wickens, 1984). As a result, depending on how much performance at those 
activities suffers, overall performance may be affected as well. With respect to the 
overall performance of the subjects during the math sessions, the fact that the original 
subject group experienced a decrease in their mean perfonnance would be supported by 
this finding. That those who used the enhanced version did not suffer would suggest 
two hypotheses. The first is that these enhancements were able to reduce the amount of 
resources necessary to perform the task so that the presence of the math problems did 
not create a lack of available resources. The second is that the additional task of doing 
math problems was not complex or difficult enough for those using the enhanced 
display and as a result, did not require many cognitive resources. When performing this 
type of experiment in the future, it will be best to make sure that the additional task 
presented to the users is of sufficient difficulty to ensure the need for extra physical or 
cognitive resources. That way, the true effectiveness of the enhancements can be 
learned. 

Overall performance was affected when the subject groups switched displays 
because some of the enhancements significantly affected how a subject performed the 
task. Two of the enhancements that the enhanced-to-original group did not have 
advantage of were associated with action-performance (String Color and Purple 
Highlight-Orbiting). The actions associated with these display changes had been rated 
as difficult and therefore it is not surprising to notice that performance suffered without 
their aid. The action-selection enhancement of the Blue ‘X’ also affected overall 
performance when the subjects switched displays. Thus it appears that certain action- 
selection enhancements may just as equally be missed. Since the effectiveness on 
overall performance of the remaining enhancements had not been affected by switching 



153 


displays, one can reason that those display changes were able to increase the enhanced 
groups’ performance beyond what would suffer from their absence and/or the subjects 
were able to perform the task based on other information and cues embedded within the 
display. Table 8-2 summarizes the most interesting results of this research. 

Final Conclusions 

The display enhancements incorporated into the display of the Star Cruiser 
task had a significant, positive impact on user performance during early sessions. The 
enhancements were developed from the results of an ecological task analysis performed 
on a version of Star Cruiser. Since the enhancements helped to improve the users’ 
abilities at pofcnming the task, procedures for performing the task analysis as outlined 
by the Ecological Task Analysis framework would;be considered useful, reliable, and 
valid for enhancing the display of this task. 

Future work in this area should, however, investigate the framework’s 
usefulness, reliability, and validity as it is used on other complex tasks. In addition, 
several quMtmns should be addressed. One question is the effect such enhancements 
have on task learning. Unfortunately, data from the J[aining sessions of this experiment 
were not analyzed. If they were, possible results would have shown why enhanced 
display subjects performed better right after training when compared to original display 
subjects. Another issue to consider is the type of information that should be presented 
by the enhancements. The results of an ecological task analysis focus on areas of the 
task where problems exist because information is lacking or misleading The results do 
not instmct the display designer as to what information should be presented to help 
alleviate those problems. Possible future research could include investigating how 
different types of information can affect the effectiveness of the task analysis. Besides 
the question of what information to present, the question of how to present it is also 
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Table 8-2. Summary Of Interesting Results 


Normal Sessions 

• Enhanced-display subject performance better than original- 
di^ay 

• All five action-performance enhancements contributed to improving 
enhanced-display group's performance 

• Two action- selection enhancements did also 

(Star Cruiser Capacity Gauges and Color Of Suns) 

• Three other action-selection enhancements were thought more 
beneficial during training 

(Colored Field, Background Colms, X-Out Tools) 


Math Sessions 

• Original-display subject performance suffered; Enhanced- 
display subject performance did NOT suffer 

• Two action-performance enhancements affected performance, but were not 
influenced by math problems 

(Purple Highlight - Orbiting and Purple Highlight - Docking) 

• Three action-selection enhancements and (me action-performance 
enhancement affected performance and were affected by the math problems 

^ (Ghost Images, Sun Color, Blue 'X; String Color) 


Transfer Sessicms 

• Overall subject perfomnance did NOT suffer regardless of 
display 

• Absence of two action-performance enhancements affected enhanced-to- 
original group 

(String Color, Purple Highlight-Orbiting) 

• Absence of one action-selection enhancement affected enhanced-to- 
original group 

(Blue X) 
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interesting. This research involved conveying the information via perceptual cues. 
Perhaps different perceptual cues, some which present the same information, than those 
used could have improved performance further. Or peiliaps they could have worsened 
performance. TherefOTe, investigating how the use of different cues might affect 
performance is another future research area. It is apparent that before a general 
statement covering many different types of complex tasks concerning the effectiveness 
of the Ecological Task Analysis framewoik can be made, fnrdier work in the 
aforemendoited areas should be perfexmed. It is obvious that many factors will 
contribute to the framework’s potential success. 
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PHOTOGRAPHS OF STAR CRUISER INTERFACES 
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Figure A-1. Star Cruiser Interface - Global View (Original Display ^fcrsion) 
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Figure A-2. Star Cruiser Interface - Local View (Original Display Version) 
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Figure A-3. Star Cruiser Interface - Local View / Star Cruiser In Orbit 
(Enhanced Display Vfersion) 
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Figure A-4. Star Cruiser Interface - Global View (Enhanced Display Version) 
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Figure A-5. Star Cruiser Interface - Lxx:al View (Enhanced Display \fersion) 
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Figure A-6. Star Cruiser Interface - Global View/ Star Cruiser Docked 
(Enhanced Display ^rsion) 
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APPENDIX B 

STAR CRUISER SCENARIO FILE VARIABLES 


The following is a listing of the variables present in a Star Cruiser scenario 
file. The tile is accessed by the simulation at the beginning of each session. The galaxy 
configuration, certain Star Cruiser movement variables, and other session parameters 
depend on the values listed in the tile. 

SR 

- Star Base location in the galaxy (X and Y coordinates) 

- Star Cruiser starting location in the galaxy (X and Y coordinates); 

• maximum amount of data Star Cruiser can cany; 

- maximum amount of resources Star Cruiser can cany 

S1-SI2 

- location of Solar System number x (X and Y coordinates); 

- age of the solar system's sun; 

- radius of the solar system's sun; 

- number of planets in the solar system 
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Pl-£6 

- location of Planet x in solar system (X and Y coordinates); 

- orbital number of the planet; 

- amount of data available on the planet; 

- amount of resources available on the planet; 

- whether or not planet supports life 

£E 

- number of PRobes onboard Star Cruiser at start of simulation 

SI 

- number of Satellites onboard Star Cruiser at start of simulation 

ss 

- number of Science Ships onboard Star Cruiser at start of simulation 
RM 

- number of Robot Miners onboard Star Cruiser at start of simulation 
MS 

- number of Miner Ships onboard Star Cruiser at start of simulation 
PDP 

- Points scored fOT each Data Package returned to Star Base 

ERC 

- Points scored for each Resources package returned to Star Base 
FCR 

- Fuel Consumption Rate 

El 

- Planet Toggle - determines whether or not information about planets' 
commodities in a solar system is displayed on the sun in the global map 
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Q1 

- Orbital Speed of planets 

OS. 

- Gain attributed to the Sun's pull 

- DeadBand on the mouse movement 

MI 

- Minimum Inertia - used to determine Star Cruiser's momentum 

IS 

- Track Scale • gain on the track (actual path of movement) of Star Cruiser 

- Force Scale • gain on the force applied with a pull of the thrust string 

EY 

- Entry Velocity - maximum allowable velocity of Star Cruiser for it to obtain 
orbit 

QEL 

- (Drbital Buffer - number of pixels from the orbit that the midpoint of Star 
Cruiser must be within before cruiser obtains orbit 

BQF 

- Break Orbit Force • BOF value times mass (radius) of sun equals force 
required to remove Star Cruiser from orbit around that sun 

LM 


- Length of Mission (minutes) 
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APPENDIX C 


ANALYSIS OF VARIANCE TABLES 


NORMAL SESSIONS 


OVERALL SCORE 


Source 

QE 

Slim nf .Sqiiaw>« 

Mean Squares 

EYaluc 

Pr>F 

SESSION 

11 

896188239.92 

81471658.17 

3.55 

0.0002 

DISPLAY 

1 

229228354.69 

229228354.69 

10.00 

0.0019 

SUBJECT 

15 

916220832.33 

61061388.82 

2.67 

0.0013 

SESSION*DISPLAY 

11 

142587344.81 

12%2485.89 

0.57 

0.8543 

Error 

153 

3506523148.9 

22918452.0 



Corrected Total 

191 

5690747920.7 





SELECT WRONG TOOL 


Source 

DE 



F Value 

Pr>F 

SESSION 

11 

0.10416667 

0.00946970 

1.01 

0.4421 

DISPLAY 

1 

0.02083333 

0.02083333 

2.22 

0.1385 

SUBJECT 

15 

OJ 1250000 

0.02063333 

2.22 

0.0079 

SESS10N*DISPLAY 

11 

0.10416667 

0.00946970 

1.01 

0.4421 

Error 

153 

1.43750000 

0.00939542 



Corrected Total 

191 

1.97916667 





SELECTION ACTION ERROR 


Source 

I2E 

Sum of Squares 

Mean Squares 

F Value 

Pr>F 

SESSION 

11 

12.43229167 

1.13020633 

0.81 

0.6301 

DISPLAY 

1 

1.50520833 

1.50520833 

1.08 

0.3006 

SUBJECT 

15 

25.86979167 

1.72465278 

1.24 

0.2509 

SESSION*DISPLAY 

11 

15.30729167 

1J9157197 

1.00 

0.4515 

Error 

153 

213.50520833 

1.39545888 



Corrected Total 

191 

268.61979167 






168 


DEPLOYMENT / RETREVAL ERROR 


Source 

D£ 

Slim nf Square 

Mean Squares 

F Value 

Pr>F 

SESSION 

11 

754.291667 

68.571970 

2.43 

0.0080 

DISPLAY 

1 

3104.083333 

3104.083333 

110.22 

0.0001 

SUBJECT 

15 

13157.916667 

877.194444 

31.15 

0.0001 

SESSION*DISPLAY 

11 

540.791667 

49.162879 

1.75 

0.0684 

Error 

153 

4308.833333 

28.162309 



Corrected Total 

191 

21865.916667 





TOOLS OUT OF COMMISSION 


Source 

DE 

Sum nf Sr^iiarea 

Mean Squares 

F Value 

Pr>F 

SESSION 

11 

24.18750000 

2.19886364 

2.88 

0.0018 

DISPLAY 

1 

0.52083333 

0.52083333 

0.68 

0.4099 

SUBJECT 

15 

18.81250000 

1.25416667 

1.64 

0.0682 

SESSION*DISH.AY 

11 

3.10416667 

0.28219697 

0.37 

0.9658 

Error 

153 

116.68750000 

0.76266340 



Corrected Total 

191 

163.31250000 





UTILIZATION OF STAR CRUISER CAPACITY 


Source 

DE 


MeanSouaiea 

F Value 

Pt>F 

SESSION 

11 

0.51641392 

0.04694672 

2.71 

0.0035 

DISPLAY 

1 

0.13383315 

0.13383315 

7.74 

0.0062 

SUBJECT 

14 

1.95561891 

0.13968706 

8.07 

0.0001 

SESSION*DISPLAY 

11 

0.46452340 

0.04222940 

2.44 

0.0083 

Error 

133 

2.30094987 

0.01730037 



Corrected Total 

170 

5.37133924 





TIME TO ORBIT 


Source 

DE 

Slim nf SqiianBit 

Mean Sgiuuw* 

F Value 

Pr;>E 

SESSION 

11 

712.7425870 

64.7947806 

2.02 

0.0302 

DISPLAY 

1 

2278.3763070 

2278.3763070 

70.93 

0.0001 

SUBJECT 

15 

4954.9391598 

330.3292773 

10.28 

0.0001 

SESSION*DISPLAY 

11 

250.6358256 

22.7850751 

0.71 

0.7283 

Error 

153 

4914.6544900 

32.1219248 



Corrected Total 

191 

13111.3483694 
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THRUSTS TO ORBIT 

Source 

QE 

Sum of Souarea 

Mean Sguares 

E-Yaluc 

Pr>F 

SESSION 

11 

114.1872979 

10.3806634 

0.86 

0.5852 

DISPLAY 

1 

1352.7894937 

1352.7894937 

111.49 

0.0001 

SUBJECT 

15 

3259.9653378 

217.3310225 

17.91 

0.0001 

SESSION*DISPLAY 

11 

106.3929894 

9.6720899 

0.80 

0.6427 

Error 

153 

1856.4544087 

12.1336889 



Corrected Total 

191 

6689.7895275 




THRUSTS TO DOCK 

Source 

DE 

Sum of Squarea 

Mean Souarea 

F Value 

Pr>F 

SESSION 

11 

20.81844219 

1.89258565 

2.34 

0.01 14 

DISPLAY 

1 

27.71658441 

27.71658441 

34.29 

0.0001 

SUBJECT 

14 

60.75254976 

4.33946784 

5.37 

0.0001 

SESSION*DISPLAY 

11 

15.11829703 

1.37439064 

1.70 

0.0797 

Error 

133 

107.49208558 

0.80821117 



Corrected Total 

170 

231.89795897 




TIME TO DOCK 

Source 

J2E 

Sum of Sauarea 

Mean Souarea 

F Value 


SESSION 

11 

178.26970824 

16.20633711 

3.87 

0.0001 

DISPLAY 

1 

121.04590033 

121.04590033 

28.91 

0.0001 

SUBJECT 

14 

215.55246661 

15.39660476 

3.68 

0.0001 

SESSION*DISPLAY 

11 

75.68631837 

6.88057440 

1.64 

0.0936 

Error 

133 

556.83682301 

4.18674303 



Corrected Total 

170 

1147.39121656 




COMMODITIES IN VISITED SOLAR SYSTEMS 




Source 

12E 

Sum of Souarea 

MeanSaiana 

F Value 

Pr>F 

SESSION 

11 

738728.4148 

67157.1286 

8.44 

0.0001 

DISPLAY 

1 

169331.4636 

169331.4636 

21.28 

0.0001 

SUBJECT 

15 

1067153.4404 

71143.5627 

8.94 

0.0001 

SESSION*DISPLAY 

11 

153681.1526 

13971.0139 

1.76 

0.0664 

Error 

153 

1217397.5595 

7956.8468 



Corrected Total 

191 

3346292.0309 
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EMPTY SOLAR SYSTEMS VISITED 


Source 


Sum of Squares 

Mean Squares 

F Value 

Pr>F 

SESSION 

11 

20.93229167 

1.90293561 

2.10 

0.0232 

DISPLAY 

1 

2.29687500 

2.29687500 

2.54 

0.1133 

SUBJECT 

15 

29.61979167 

1.97465278 

2.18 

0.0091 

SESSION*DISPLAY 

11 

17.01562500 

1.54687500 

1.71 

0.0762 

Error 

153 

138.50520833 

0.90526280 



Corrected Total 

191 

208.36979167 




ERRORS IN DETERMINING STAR CRUISER MODE 



Source 

12E 

Sum of Squares 

McanSguam 

F Value 

Pr>F 

SESSION 

11 

0.15430967 

0.01402815 

2.06 

0.0263 

DISPLAY 

1 

0.05105024 

0.05105024 

7.51 

0.0069 

SUBJECT 

15 

0.30097185 

0.02006479 

2.95 

0.0004 

SESSION*DISPLAY 

11 

0.15152666 

0.01377515 

2.03 

0.0295 

Error 

153 

1.04070271 

0.00680198 



Corrected Total 

191 

1.69856113 
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MATH SESSIONS 


OVERALL SCORE 


Source 

HE 

Sum of Squares 

Mean Squares 

F Value 

El^ 

CONDmON 

1 

41812389.06 

41812389.06 

2.05 

0.1598 

DISPLAY 

1 

112503145.56 

112503145.56 

5.51 

0.0236 

SUBJECT 

15 

969599260.00 

64639950.67 

3.16 

0.0016 

SESSION (CONDITION) 

2 

77091470.31 

38545735.16 

1.89 

0.1639 

CONDITION*DISPLAY 

1 

43573201.00 

43573201.00 

2.13 

0.1514 

Error 

43 

878474404.1 

20429637.3 



Corrected Total 

63 

2123053870.0 




SELECT WRONG TOOL 





Source 

DE 

Sum of Squarea 

Mean Squares 

F Value 

Pr>F 

CONDITION 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

DISPLAY 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

SUBJECT 

15 

0.23437500 

0.01562500 

1.00 

0.4726 

SESSION (CONDITION) 

2 

0.03125000 

0.01562500 

1.00 

0.3763 

CONDmON*DISPLAY 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

Error 

43 

0.67187500 

0.01562500 



Corrected Total 

63 

0.98437500 




SELECTION ACTION ERROR 





Source 

HE 



F Value 

Pr>F 

CONDITION 

1 

5.06250000 

5.06250000 

1.43 

0.2388 

DISPLAY 

1 

4.00000000 

4.00000000 

1.13 

0.2943 

SUBJECT 

15 

59.43750000 

3.96250000 

1.12 

0.3711 

SESSION (CONDITION) 

2 

3.12500000 

1.56250000 

0.44 

0.6467 

CONDITION*DISPLAY 

1 

6.25000000 

625000000 

1.76' 

0.1914 

Error 

43 

152.56250000 

3.54796512 



Corrected Total 

63 

230.43750000 
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DEPLOYMENT / RETRIEVAL ERROR 


Source 

CE 

Sum of Squares 

Mean Squares 

F Value 

El>£ 

CONDITION 

1 

118.2656250 

118.2656250 

99999.99 

0.0001 

DISPLAY 

1 

2613.7656250 

2613.7656250 

99999.99 

0.0001 

SUBJECT 

15 

8970.9843750 

598.0656250 

99999.99 

0.0001 

SESSION (CONDITION) 

2 

120.1562500 

60.0781250 

99999.99 

0.0001 

CONDinON*DISPLAY 

1 

50.7656250 

50.7656250 

99999.99 

0.0001 

Error 

43 

0.000000 

0.000000 



Corrected Total 

63 

11414.234375 




TOOLS OUT OF COMMISSION 





Source 

DE 

Sfm ff 

Mean Squares 

EYaluc 

El>^ 

CONDITION 

1 

1.56250000 

1.56250000 

4.03 

0.0511 

DISPLAY 

1 

1.56250000 

1.56250000 

4.03 

0.0511 

SUBJECT 

15 

7.93750000 

0.52916667 

1.36 

0.2090 

SESSION (CONDITION) 

2 

0.62500000 

0.31250000 

0.81 

0.4536 

CONDinON*DISPLAY 

1 

0.06250000 

0.06250000 

0.16 

0.6902 

Error 

43 

16.68750000 

0.38808140 



Corrected Total 

63 

28.43750000 




UTILIZATION OF STAR CRUISER CAPACITY 




Source 

CE 

.^iim nf Sqiimea 

McanSffliarea 

EYalufi 

EL>i 

CONDITION 

1 

0.33012410 

033012410 

1.34 

0.2535 

DISPLAY 

1 

0.15413968 

0.15413968 

0.63 

0.4332 

SUBJECT 

14 

Z00210688 

0.14300763 

0.58 

0.8626 

SESSION (CONDITION) 

2 

0.73730791 

0.36865395 

1.50 

0.2357 

CONDinON*DISPLAY 

1 

0.12192239 

0.12192239 

0.50 

0.4854 

Error 

39 

9.58531392 

014577728 



Corrected Total 

58 

1193091488 




TIME TO ORBIT 






Source 

J2E 

■Sum nf .Sf^iiaiea 

Mean Sgiuwe* 

F Value 

Pr>F 

CONDITION 

1 

2437283.159 

2437283.159 

0.96 

0.3320 

DISPLAY 

1 

2624262.397 

2624262.397 

1.04 

0.3143 

SUBJECT 

14 

34793577.226 

2485255316 

0.98 

0.4871 

SESSION (CONDITION) 

2 

4941814.223 

2470907.112 

0.98 

0.3850 

CONDmON*DISPLAY 

1 

2660093.108 

2660093.108 

1.05 

0.3110 

Error 

43 

108850984.765 

2531418150 



Corrected Total 

62 

156308014.879 
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THRUSTS TO ORBIT 






Source 

I2E 

Sum of Squares 

Mean Squares 

F Value 


CONDITION 

1 

0.9978095 

0.9978095 

0.07 

0.7866 

DISPLAY 

1 

623.8406137 

623.8406137 

46.38 

0.0001 

SUBJECT 

14 

1168.7984024 

83.4856002 

6.21 

0.0001 

SESSION (CONDITION) 

2 

18.2776636 

9.1388318 

0.68 

0.5122 

CONDmON*DISMAY 

1 

3.8632106 

3.8632106 

0.29 

0.5948 

Error 

43 

578.3309866 

13.4495578 



Corrected Total 

62 

2394.1086865 




THRUSTS TO DOCK 






Source 

DE 

Sum of Squares 

Mmii 

F Value 

Pr>F 

CONDITION 

1 

1.47545356 

1.47545356 

1.73 

0.1962 

DISPLAY 

1 

4.15715456 

4.15715456 

4.87 

0.0332 

SUBJECT 

14 

26.84398722 

1.91742766 

2.25 

0.0234 

SESSION (CONDITION) 

2 

4.42280703 

2.21140352 

2.59 

0.0877 

CONDinON*DISPLAY 

1 

0.12259250 

0.12259250 

0.14 

0.7067 

Error 

39 

33.27760568 

0.85327194 



Corrected Total 

58 

70.29960056 




TIME TO DOCK 






Source 

BE 

SumofSflMm 

Mmwi SqiutTBa 

EYalufi 


CONDITION 

1 

6646.034909 

6646.034909 

0.98 

0.3282 

DISPLAY 

1 

5533.948720 

5533.948720 

0.82 

0.3718 

SUBJECT 

14 

84829.871344 

6059.276525 

0.89 

0.5715 

SESSION (CONDITION) 

2 

13138.854150 

6569.427075 

0.97 

0.3884 

CONDITION*DISPLAY 

1 

6574.348874 

6574.348874 

0.97 

0.3308 

Error 

39 

264387.93399 

6779.17779 



CtOTocted Total 

58 

381110.99199 




COMMODITIES IN VISITED SOLAR SYSTEMS 




Source 

12E 


Mean Sgiiarn* 

F Value 

Pr>F 

CONDITION 

1 

113296.85518 

113296.85518 

13.77 

0.0006 

DISPLAY 

1 

166221.35287 

166221.35287 

20.20 

0.0001 

SUBJECT 

15 

934012.62614 

62267.50841 

7.57 

0.0001 

SESSION (CONDITION) 

2 

178538.36599 

89269.18300 

10.85 

0.0002 

CONDinON*DISPLAY 

1 

13098.83652 

13098.83652 

1.59 

0.2139 

Error 

43 

353921.8951 

8230.7417 




Corrected Total 


63 


1759089.9318 
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EMPTY SOLAR SYSTEMS VISITED 


Source 

Q£ 

Slim of SqimrBg 

MeanSouarea 

F Value 

Pr>F 

CONDITION 

1 

3.51562500 

3.51562500 

8.55 

0.0055 

DISPLAY 

1 

0.01562500 

0.01562500 

0.04 

0.8463 

SUBJECT 

15 

6.73437500 

0.44895833 

1.09 

0.3911 

SESSION (CONDITION) 

2 

1.53125000 

0.76562500 

1.86 

0.1675 

CONDmON*DISPLAY 

1 

0.01562500 

0.01562500 

0.04 

0.8463 

Error 

43 

17.67187500 

0.41097384 



Corrected Total 

63 

29.48437500 





ERRORS IN DETERMINING STAR CRUISER MODE 

SfimCfi QE Sum of Squarea MeanSquarea F Value Pr>F 


CCWOmON 1 

DISPLAY 1 

SUBJECT 15 

SESSIC»I (CONDITION) 2 

CONDmON*DISPLAY 1 


0.00078267 0.00078267 

0.00000142 0.00000142 

0.06679947 0.00445330 

0.00833615 0.00416808 

0.00238237 0.00238237 


0.17 0.6811 

0.00 0.9860 

0.97 0.4971 

0.91 0.4095 

0.52 0.4743 


Error 

43 

0.19660095 

0.00457212 

Corrected Total 

63 

0.27490303 
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TRANSFER SESSIONS 


OVERALL SCORE 


Source 


Sum of Squares 

Mean Squares 

EXaluc 


CONDITION 

1 

97088998.89 

97088998.89 

3.81 

0.0574 

DISPLAY 

1 

21547003.52 

21547003.52 

0.85 

0.3628 

SUBJECT 

15 

585383172.61 

39025544.84 

1.53 

0.1362 

SESSION (CONDITION) 

2 

92482025.78 

46241012.89 

1.82 

0.1749 

CONDmON*DISMAY 

1 

404655.02 

404655.02 

0.02 

0.9003 

Error 

43 

1094975280.55 

25464541.41 



Corrected Total 

63 

1891881136.36 




SELECT WRONG TOOL 





Source 

CE 

Sum of Sauarea 

McanSfluam 

F Value 

Pr>F 

CONDITION 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

DISPLAY 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

SUBJECT 

15 

0.23437500 

0.01562500 

1.00 

0.4726 

SESSION (CONDITION) 

2 

0.03125000 

0.01562500 

1.00 

0.3763 

CONDinON'DISPLAY 

1 

0.01562500 

0.01562500 

1.00 

0.3229 

Error 

43 

0.67187500 

0.01562500 



Corrected Total 

63 

0.98437500 




SELECTION ACTION ERROR 





Source 

QE 



F Value 

Pr>F 

CONDITION 

1 

3.51562500 

3.51562500 

0.95 

0.3352 

DISPLAY 

1 

6.89062500 

6.89062500 

1.86 

0.1796 

SUBJECT 

15 

58.98437500 

3.93229167 

1.06 

0.4167 

SESSION (CONDITION) 

2 

3.15625000 

1.57812500 

0.43 

0.6556 

CONDmON*DISPLAY 

1 

3.51562500 

3.51562500 

0.95 

0.3352 

Error 

43 

159.17187500 

3.70167151 



Corrected Toad 

63 

235.23437500 
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DEPLOYMENT / RETRffiVAL ERROR 


Source 

D£ 

Sum of Sqimrea 

Mean Squares 

LYaiufi 

Pr>F 

CONDITION 

1 

14.0625000 

14.0625000 

0.20 

0.6579 

DISPLAY 

1 

729.0000000 

729.0000000 

10.31 

0.0025 

SUBJECT 

15 

3712.5000000 

247.5000000 

3.50 

0.0006 

SESSION (CONDITION) 

2 

176.5625000 

88.2812500 

1.25 

0.2972 

CONDmON*DISPLAY 

1 

976.5625000 

976.5625000 

13.81 

0.0006 

Enor 

43 

3041.3125000 

70.7281977 



Corrected Total 

63 

8650.0000000 




TOOLS OUT OF COMMISSION 





Source 

C£ 

SumofSouarea 

MeanSouarea 

F Value 

Pr>F 

CONDmON 

1 

0.14062500 

0.14062500 

0.37 

0.5472 

DISPLAY 

1 

0.39062500 

0.39062500 

1.02 

0.3175 

SUBJECT 

15 

8.73437500 

0.58229167 

1.52 

0.1390 

SESSION (CONDITION) 

2 

1.65625000 

0.82812500 

2.17 

0.1267 

CONDITION*DISPLAY 

1 

0.14062500 

0.14062500 

0.37 

0.5472 

Error 

43 

16.42187500 

0.38190407 



Ctxrected Total 

63 

27.48437500 




UTILIZATION OF STAR CRUISER CAPACITY 




Source 

J2E 

SumofSfluarea 

MeanSouarea 

F Value 

Pr>F 

CONDITION 

1 

0.00352547 

0.00352547 

0.02 

0.8961 

DISPLAY 

1 

0.06454589 

0.06454589 

0.32 

0.5771 

SUBJECT 

14 

3.16092843 

0.22578060 

1.11 

0.3825 

SESSION (CONDITION) 

2 

0.77702948 

0.38851474 

1.90 

0.1625 

CONDmON*DISPLAY 

1 

0.21246067 

0.21246067 

1.04 

0.3139 

Error 

40 

8.16849239 

0.20421231 

• 


Corrected Total 

59 

12.38698233 




TIME TO ORBIT 






Source 

QE 

SumofSouarea 

MeanSouarea 

F Value 


CONDITION 

1 

2473512.786 

2473512.786 

1.00 

0.3236 

DISPLAY 

1 

2501637.564 

2501637 J64 

1.01 

0.3209 

SUBJECT 

15 

37199255.447 

2479950 J63 

1.00 

0.4728 

SESSION (CONDITION) 

2 

4939453.107 

2469726354 

1.00 

0.3778 

CONDITION*DISPLAY 

1 

2570495.717 

2570495.717 

1.04 

0.3144 

Error 

43 

106659701.478 

2480458.174 



Corrected Total 

63 

156344056.099 
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THRUSTS TO ORBIT 






Source 

I 2 E 

Sum rtf .Sqiiflies 

Mean Squares 

EYalufi 

Pr>F 

CONDITION 

1 

3.93550807 

3.93550807 

0.21 

0.6515 

DISPLAY 

1 

33.62524642 

33.62524642 

1.77 

0.1907 

SUBJECT 

15 

612.16721153 

40.81114744 

2.14 

0.0258 

SESSION (CONDITION) 

2 

13.30210528 

6.65105264 

0.35 

0.7070 

CONDITION*DISPLAY 

1 

434.95495644 

434.95495644 

22.86 

0.0001 

Error 

43 

818.1387324 

19.0264821 



Conected Total 

63 

1916.1237601 




THRUSTS TO DOCK 






Source 

12 E 

Sum of Sauarea 

Mean Squares 

EJ^aiue 

EL2_E 

CONDITION 

1 

0.16540317 

0.16540317 

0.18 

0.6763 

DISPLAY 

1 

5.22633117 

5.22633117 

5.59 

0.0230 

SUBJECT 

14 

29.42121472 

2.10151534 

2.25 

0.0228 

SESSION (CONDITION) 

2 

1.85255800 

0.92627900 

0.99 

0.3803 

CONDmON*DISPLAY 

1 

0.06224565 

0.06224565 

0.07 

0.7977 

Error 

40 

37.40237432 

0.93505936 



Corrected Total 

59 

74.13012703 




TIME TO DOCK 






Source 

12 E 

SumofSqiMTM 

Mean Squares 

F Value 

E[^ 

CONDITION 

1 

3388.22965 

3388.22%5 

0.13 

0.7190 

DISPLAY 

1 

5784.08198 

5784.06198 

0.22 

0.6385 

SUBJECT 

14 

328306.93118 

23450.49506 

0.91 

0.5571 

SESSION (CONDITION) 

2 

56082.35302 

28041.17651 

1.09 

0.3472 

CONDinON*DISPLAY 

1 

60634.34333 

60634.34333 

2.35 

0.1332 

Error 

40 

1032443.83783 

25811.09595 



Corrected Total 

59 

1486639.77699 




COMMODITIES IN VISITED SOLAR SYSTEMS 




Source 

C£ 

Sum of Squares 

Mwm Sqm>w»« 

EYalus 

El^ 

CONDITION 

1 

880.0670 

880.0670 

0.07 

0.7945 

DISPLAY 

1 

94546.2749 

94546.2749 

7.38 

0.0095 

SUBJECT 

15 

1052639.3280 

70175.9552 

5.48 

0.0001 

SESSION (CONDITION) 

2 

219348.9610 

109674.4805 

8.56 

0.0007 

CONDmON*DISPLAY 

1 

46082.7360 

46082.7360 

3.60 

0.0646 


Error 

43 

550973.0080 

12813.3258 

Corrected Total 

63 

1964470.3749 
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EMPTY SOLAR SYSTEMS VISITED 


Source 

I2E 

SumofSouaiea 

Mean Squares 

F Value 

Pr>F 

CONDITION 

1 

2.64062500 

2.64062500 

5.96 

0.0188 

DISPLAY 

1 

0.01562500 

0.01562500 

0.04 

0.8519 

SUBJECT 

15 

7.23437500 

0.48229167 

1.09 

0.3941 

SESSION (CONDITION) 

2 

1.53125000 

0.76562500 

1.73 

0.1897 

CONDmON*DISPLAY 

1 

0.01562500 

0.01562500 

0.04 

0.8519 

Error 

43 

19.04687500 

0.44295058 



Corrected Total 

63 

30.48437500 




ERRORS IN DETERMINING STAR CRUISER 

MODE 



Source 

12E 

Slim nf Sgiiarwa 

McaoSgiiaitt 

F Value 

Pr>F 

CONDITION 

1 

0.02189534 

0.02189534 

2.43 

0.1266 

DISPLAY 

1 

0.00730525 

0.00730525 

0.81 

0.3733 

SUBJECT 

15 

0.12005687 

0.00800379 

0.89 

0.5827 

SESSION (CONDITION) 

2 

0.01791007 

0.00895503 

0.99 

0.3790 

CONDmON*DISPLAY 

1 

0.01771288 

0.01771288 

1.96 

0.1684 

Error 

43 

0.38803143 

0.00902399 



Corrected Total 

63 

0.57291185 
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